1. Monocular Simultaneous Localizationand Generalized Object Mapping withUndelayed Initialization 1Robot Perception and Learning Lab 2010/7/23

2. Outline Introduction State Vector Definition Proposed Classification Algorithm Simulations and Real experiment Conclusion2 Robot Perception and Learning Lab 2010/7/23 3. Outline Introduction State Vector Definition Proposed Classification Algorithm Simulations and Real experiment Conclusion3 Robot Perception and Learning Lab 2010/7/23 4. EKF-based SLAM4 Robot Perception and Learning Lab 2010/7/23 5. Monocular SLAM Camera as the only sensor Andrew J. Davison et al. proposed a EKF-based SLAM approach Andrew J. Davison, Ian Reid, Nicholas Molton and Olivier Stasse:MonoSLAM: Real-Time Single Camera SLAM, IEEE TRANSACTIONSON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.29, NO. 6, JUNE 2007 Feature states are the 3D position vectors of the locations of point features Multiple images acquired must be combined to achieve accurate depth estimates5 Robot Perception and Learning Lab2010/7/23 6. Inverse Depth Parametrization6 Robot Perception and Learning Lab 2010/7/23 7. Inverse Depth Parametrization High degree oflinearity Ability to cope withfeatures far from thecamera UndelayedinitializationJ.M.M. Montiel, Javier Civera and Andrew J. Davison:Unified Inverse Depth Parametrization for Monocular SLAM.Robotics: Science and Systems Conference 2006.7 Robot Perception and Learning Lab2010/7/23 8. Dynamic Environments Inclusion of moving features => Degrade performance Prior knowledge Avoid moving objects SomkiatWangsiripitak, David W. Murray: Avoiding moving outliers invisual SLAM by tracking moving objects, ICRA20098 Robot Perception and Learning Lab2010/7/23 9. Performance DegradeSLAM with static featuresSLAM with static features and a new moving feature Chieh-ChihWang, Ko-Chih Wang, Chen-Han Hsiao, Kuen-Han Lin and Yi-Liu Chao.: Monocular Vision-based Simultaneous Localization, Mapping and Moving Object Tracking, submit to journal9 Robot Perception and Learning Lab 2010/7/23 10. Classification Stage Accumulate temporal information Multiple imagesrequired Abnormal negative inverse depthChieh-ChihWang, Ko-Chih Wang, Chen-Han Hsiao, Kuen-Han Lin and Yi-Liu Chao.:Monocular Vision-based Simultaneous Localization, Mapping and Moving Object Tracking,submit to journal10 Robot Perception and Learning Lab 2010/7/23 11. Contributions in this thesis SLAM with generalized objects Proposed parametrization Undelayed Initialization Classification algorithm based on velocity11 Robot Perception and Learning Lab 2010/7/23 12. Outline Introduction State Vector Definition Proposed Classification Algorithm Simulations and Real experiment Conclusion12 Robot Perception and Learning Lab 2010/7/23 13. State vector definition EKF-based SLAM with generalized objects State vector: Camera: Generalized object: The inverse depth parametrization Proposed parametrization With motion model13 Robot Perception and Learning Lab 2010/7/23 14. Dynamic Inverse Depth Parametrization 9-dimension state value Position Velocity 3D Location w.r.t. XYZ coordinate system:14 Robot Perception and Learning Lab 2010/7/23 15. Feature parametrization and motion prediction15 Robot Perception and Learning Lab 2010/7/23 16. Dynamic Inverse Depth Parametrization Motion Predict(constant velocity assumption)16 Robot Perception and Learning Lab2010/7/23 17. Measurement Model The observation of a point feature17 Robot Perception and Learning Lab2010/7/23 18. Undelayed Feature Initialization Initialized using only one image First observed frame18 Robot Perception and Learning Lab2010/7/23 19. Initial value of inverse depth and velocity Initial value of inverse depth Range of depth: [ d min , ] 1 Range of inverse depth: [ 0 ,] d min To cover its 95% acceptance region: 0 11 ,2 d min2 d min Initial value of velocity Range of velocity: [ | v | max , | v | max ] To cover its 95% acceptance region: v 0 | v | max 0, v 219 Robot Perception and Learning Lab 2010/7/23 20. Outline Introduction State Vector Definition Proposed Classification Algorithm Simulations and Real experiment Conclusion20 Robot Perception and Learning Lab 2010/7/23 21. Classification Cue Conduct simulation to show the convergency of velocity 3 targets in the simulation Coded in dynamic inverse depth parametrization21 Robot Perception and Learning Lab 2010/7/23 22. Classification Cue: Velocity Convergency22 Robot Perception and Learning Lab 2010/7/23 23. Classification Cue: Velocity Convergency23 Robot Perception and Learning Lab 2010/7/23 24. Classification Cue: Velocity Convergency24 Robot Perception and Learning Lab 2010/7/23 25. Score function for classifying static objects Given the velocity distribution: Probability density function value of the velocity distribution at The relative likelihood at Classification by thresholding Cs(X )ts => classify as static object25 Robot Perception and Learning Lab2010/7/23 26. Threshold selection on ts PDF value of static object at is expected higher Threshold selection t s26 Robot Perception and Learning Lab 2010/7/23 27. Score function for classifying moving objects Given the velocity distribution: Mahalanobis distance function The velocity of moving objects is expected to convergeaway from Classification by thresholding Cm (X )tm => classify as moving object27 Robot Perception and Learning Lab2010/7/23 28. Threshold selection on tm M-dist of a moving object at is expected to larger Threshold selection t m28 Robot Perception and Learning Lab2010/7/23 29. Classification StateInitialized feature Generalized objects instate vector Unknown state Unknownstate Static state Moving state Cs(X ) ts Cm (X ) tm Low computationalclassificationStatic Movingalgorithm state state29 Robot Perception and Learning Lab 2010/7/23 30. State transition Unknown state to Unknown state to Static state Moving state Change the label Change the label Adjust values to Keep the same valuessatisfied the propertyv0, v0SLAM with generalized object is achieved.30 Robot Perception and Learning Lab2010/7/23 31. Issue on unobservable situations disability of monocular system to find an unique trajectory of an object under the constant-velocity assumption31 Robot Perception and Learning Lab 2010/7/23 32. Issue on unobservable situations Conduct simulation to show the convergency of velocity 3 targets in the simulation Coded in dynamic inverse depth parametrization32 Robot Perception and Learning Lab 2010/7/23 33. Ambiguation under unobservable situations33 Robot Perception and Learning Lab 2010/7/23 34. Ambiguation under unobservable situations Cannot distinguish the state according to the velocity distribution Ambiguation of Static object Constant speedparallel-moving object34 Robot Perception and Learning Lab2010/7/23 35. Non-parallel moving object under unobservable situations35 Robot Perception and Learning Lab 2010/7/23 36. Non-parallel moving object under unobservable situations 95% confidence region do not cover (0,0,0) No ambiguation with static object36 Robot Perception and Learning Lab2010/7/23 37. Classification under unobservable situations Ambiguation Static object Constant speed parallel-moving object Non ambiguation Constant speed non parallel-moving object37 Robot Perception and Learning Lab 2010/7/23 38. Outline Introduction State Vector Definition Proposed Classification Algorithm Simulations and Real experiment Conclusion38 Robot Perception and Learning Lab 2010/7/23 39. Simulation Observable situationUnobservable situation39 Robot Perception and Learning Lab 2010/7/23 40. Simulation setting 50 Monte Carlo simulations 300 static landmarks and 288 moving landmarks In each simulation40 Robot Perception and Learning Lab2010/7/23 41. Classification error ratio Observable situationUnobservable situation41 Robot Perception and Learning Lab 2010/7/23 42. Classification Result Classification Result of 50 Monte Carlosimulations under threshold Observable situation42 Robot Perception and Learning Lab 2010/7/23 43. Convergency of SLAM with generalized objects Convergency of camera43 Robot Perception and Learning Lab 2010/7/23 44. Convergency of SLAM with generalized objects Convergency of static objects44 Robot Perception and Learning Lab 2010/7/23 45. Convergency of SLAM with generalized objects Convergency of moving objects45 Robot Perception and Learning Lab 2010/7/23 46. Real Experiment NTU PAL7 robot Wide-angle camera 79.48 degree view angle 640 480 Laser scanner At the basement of CSIE 1793 images,13.65fps, 131 seconds46 Robot Perception and Learning Lab 2010/7/23 47. Real Experiment at basement of CSIE47 Robot Perception and Learning Lab 2010/7/23 48. Real Experiment Classification result48 Robot Perception and Learning Lab 2010/7/23 49. Video of Real Experiment49 Robot Perception and Learning Lab 2010/7/23 50. Comparison of estimation and ground-truth (Topview)50 Robot Perception and Learning Lab 2010/7/23 51. Comparison of estimation and ground-truth (Sideview)51 Robot Perception and Learning Lab 2010/7/23 52. Outline Introduction State Vector Definition Proposed Classification Algorithm Simulations and Real experiment Conclusion52 Robot Perception and Learning Lab 2010/7/23 53. Conclusions Achieve SLAM with generalized objects Simulations Real experiments Adopt un-delayed initialization Provide a low computational classification algorithm Competitive performance to laser-based approach53 Robot Perception and Learning Lab2010/7/23