research on 3d model retrieval pu jiantao. [email protected] october, 2003. department of...

Research On 3D Model Research On 3D Model RetrievalRetrieval

Pu Jiantao.Pu [email protected]@cis.pku.edu.cn

October, 2003.October, 2003.

Department of Intelligent Science, Peking UniversityDepartment of Intelligent Science, Peking University

版权所有，未经允许，禁止使用

ContentContent

• DefinitionDefinition• MotivationMotivation• DifficultyDifficulty• Related WorksRelated Works• Our Research ObjectiveOur Research Objective• Outlines of Our MethodsOutlines of Our Methods• OrganizationOrganization• SummarySummary

Definition: Definition: what’s 3D Model what’s 3D Model RetrievalRetrieval??

• Its goal is to find out the models with the Its goal is to find out the models with the most desired geometric shape from 3D most desired geometric shape from 3D model database.model database.

Feature?

3D Model Database Query

Result

Definition: Definition: how to do queryhow to do query??

• 3D model can be describe by:3D model can be describe by: Text about 3d model’s properties, Text about 3d model’s properties, such as such as

name, color, texture, material, function, etc.name, color, texture, material, function, etc. Shape: content-base.Shape: content-base.


• Example 1: Example 1: How do you differentiate following How do you differentiate following cars precisely by the way of text?cars precisely by the way of text?


• Example 2: Example 2: Following is a model with arbitrary Following is a model with arbitrary shape, how do you describe it precisely?shape, how do you describe it precisely?


• Example 3: Example 3: The top row is a base-model, the The top row is a base-model, the bottom row is a serials of models arrayed in bottom row is a serials of models arrayed in similarity order. How do you do this similarity order. How do you do this precisely? precisely?

High Similari

ty

Low Similari

ty

Definition: Definition: conclusionconclusion??

• Key-words based query is not a good way.Key-words based query is not a good way.

• Geometric shape is the most powerful way in 3D Geometric shape is the most powerful way in 3D model description, as it is the most intuitive way model description, as it is the most intuitive way that human apperceive 3D models.that human apperceive 3D models.

So So shape matchingshape matching is the best way to do 3D model is the best way to do 3D model retrieval. However, it is a very difficult problem.retrieval. However, it is a very difficult problem.


Framework of 3D Model Retrieval System:


Main Research Targets:

• Research on Feature Definition and Extraction for 3D Models;

• Research on Similarity Measure;

• Design and Implementation of Query Interface;

• Rapid Search Methods;


• Shape matching is to extract and compare the key Shape matching is to extract and compare the key features between 3D models. features between 3D models.

Definition: Definition: the difference the difference between 3D Model Retrieval & 2D between 3D Model Retrieval & 2D

Image RetrievalImage Retrieval??

22D ImageD Image– Simple contour representation Simple contour representation

– Features occlusion, shadow, Features occlusion, shadow, noisenoise

– Camera dependentCamera dependent

– …………

33D ModelD Model– Complex surface representation Complex surface representation

– No features occlusion, shadows, No features occlusion, shadows, noisenoise

– Camera independentCamera independent

– …………..

MotivationMotivation

• 3D models can be easily obtained:3D models can be easily obtained: 3D modeler3D modeler

33D ScannerD Scanner

Rebuilt by several imagesRebuilt by several images

Commercial 3D model libraryCommercial 3D model library


• 3D models are widely used in many fields:3D models are widely used in many fields: IndustryIndustry

GamesGames

ArtArt

Virtual RealityVirtual Reality

E-businessE-business

EducationEducation

ArchitectureArchitecture


• 3D technologies are developing rapidly.3D technologies are developing rapidly.

• The number of 3D models is increasing at a The number of 3D models is increasing at a surprising rate, especially under the surprising rate, especially under the stimulation of Internet.stimulation of Internet.

DifficultiesDifficulties

• It is hard to define and extract the features of 3D It is hard to define and extract the features of 3D models;models;

• It is hard to bridge the gap between shape and It is hard to bridge the gap between shape and Semantics;Semantics;

• It is hard to measure the similarity between It is hard to measure the similarity between models;models;

• It is hard to do shape matching between models It is hard to do shape matching between models that may be in arbitrary pose.that may be in arbitrary pose.


• It is hard to define and extract the features of 3D It is hard to define and extract the features of 3D models;models;

Feature?

Feature?

Feature?


• It is hard to bridge the gap between shape and It is hard to bridge the gap between shape and Semantics;Semantics;

Similar

Hammer

Toy: DrumHow do you know it is a hammer or

drum???


• It is hard to measure the similarity between It is hard to measure the similarity between models;models;

Similar

Hammer

Toy: DrumTo which extend above models are

similar???


• It is hard to do shape matching between models It is hard to do shape matching between models that may be in arbitrary pose.that may be in arbitrary pose.


• Good features should have following characteristics:Good features should have following characteristics:– quick to computequick to compute– concise to storeconcise to store – easy to indexeasy to index– invariant under similarity transforms invariant under similarity transforms – insensitive to noise and small extra features insensitive to noise and small extra features – independent of 3D object representation independent of 3D object representation – robust to arbitrary topological degeneracies robust to arbitrary topological degeneracies – discriminating of shape similarities and differencesdiscriminating of shape similarities and differences

Related WorksRelated Works

• All methods for shape matching can be All methods for shape matching can be roughly classified as:roughly classified as:

(1) Feature vector-based;(1) Feature vector-based;

(2) Statistics-based;(2) Statistics-based;

(3) Topology-based;(3) Topology-based;

Related Works: Related Works: Feature vector-Feature vector-based Methodbased Method

• Features include:Features include:

Area, circularity, eccentricity, compactness,Area, circularity, eccentricity, compactness,

major axis orientation, Euler number, concavity, etc.major axis orientation, Euler number, concavity, etc.

Feature Vector

Database

VectorComparison

Features

Similar Models


•The idea is simple, but the implementation is The idea is simple, but the implementation is tedious. In addition, different models have tedious. In addition, different models have different key features.different key features.


•M. Hebert, K. Ikeuchi and H. Delingette. A Spherical Representation for M. Hebert, K. Ikeuchi and H. Delingette. A Spherical Representation for Recognition of Free-form Surfaces. IEEE Trans. PAMI, Vol.17, pp.681-Recognition of Free-form Surfaces. IEEE Trans. PAMI, Vol.17, pp.681-690, 1995.690, 1995.

•R. Sonthi, G. Kunjur and R. Gadh. Shape Feature Determination using the R. Sonthi, G. Kunjur and R. Gadh. Shape Feature Determination using the Curvature Region Representation. Proc. Symp. Solid Modeling, pp.285-Curvature Region Representation. Proc. Symp. Solid Modeling, pp.285-296, 1997.296, 1997.

•G. Dudek and J.K. Tsotsos. Shape Representation and Recognition from G. Dudek and J.K. Tsotsos. Shape Representation and Recognition from Multiscale Curvature. Computer Vision and Image Understanding, Vol.68, Multiscale Curvature. Computer Vision and Image Understanding, Vol.68, pp.170-189, 1997. Discrete Geodeic Problem. SIAM J. Computing, Vol.16, pp.170-189, 1997. Discrete Geodeic Problem. SIAM J. Computing, Vol.16, pp.647-667, 1987.pp.647-667, 1987.

Related Works: Related Works: Statistics-based Statistics-based MethodMethod

• Shape DistributionShape Distribution

The key idea is to represent the signature of an object as a shape distribution sampled from a shape function measuring global geometric.

Robert O., Thomas F., Bernard C., and David D., Shape Distribution. ACM Transactions on Graphics, 21(4), pp. 807-832, October 2002.


• (cont.)Shape Distribution(cont.)Shape Distribution

It is simpler than traditional shape matching methods that require pose registration, feature correspondence, or model fitting.


• (cont.) Three steps are involved:(cont.) Three steps are involved:

(1) Selecting a shape function:(1) Selecting a shape function:

A3 ： Measures the angle between three random points on the surface of a 3D model.D1 ： Measures the distance between a fixed point and one random point on the surface. Generally, the centroid of the boundary of the model is used as the fixed point.D2 ： Measures the distance between two random points on the surface.D3 ： Measures the distance between two random points on the surface.D4 ： Measures the cube root of the volume of the tetrahedron between four random points on the surface.


• (cont.)(cont.)

(1) Selecting a shape function: (cont.)(1) Selecting a shape function: (cont.)

Example D2 shape distributions.


• (cont.)(cont.)

(2) Sampling method(2) Sampling method

The time to sample a shape distribution is linearly proportional to the number of samples.

Sampling a random point in a triangle.


• (cont.)(cont.)

(2) Comparing Shape Distribution(2) Comparing Shape Distribution


• (cont.)(cont.)

(2) Comparing Shape Distribution(cont.)(2) Comparing Shape Distribution(cont.)

(1) bottleneck distance; (2) Hausdorff distance;(3) Turing function distance; (4) Fréchet Distance;(5) nonlinear elastic matching distance;

(6) reflection distance; (7) area of symmetric difference; (8) transport distance; (9) Earth Mover’s distance; (10) discrete distance;

As for detailed definitions, you can refer to: Veltkamp, R. C., Shape matching: Similarity measures and algorithms. In Shape Modelling International (Genova), 188–197, 2001.


• Ryutarou Ohbuchi, Shape-Similarity Search of Three-Dimensional Models Using Parameterized Statistics, Accepted for publication in the proceedings of the Pacific Graphics 2002, Beijing, China, October 9-11, 2002.


• (Cont.)


• (Cont.) Experiment Results:

Euclidean distance. Elastic matching.


An Example:• Sphere is evenly sampled• For each sphere sample min. distance to object calculated

Signature Comparison: L1,L2,L∞

Related Works: Related Works: Topology-based Topology-based MethodMethod

• Topology is a compact Topology is a compact representation of 3D models:representation of 3D models:

(1) Intuitive;(1) Intuitive;

(2) Flexibility: global & local;(2) Flexibility: global & local;

(3) Transform Invariance;(3) Transform Invariance;


• H. Sundar,H. Sundar, D. Silver, . Gagvani, D. Silver, . Gagvani, S. Dickinson, S. Dickinson, Skeleton Based Skeleton Based Shape Matching and RetrievalShape Matching and Retrieval, , Shape ModelingShape Modeling International International 2003, Seoul , Korea, May 12 - 2003, Seoul , Korea, May 12 -

15, 2003.15, 2003.


• (Cont.) (Cont.) Skeleton CreationSkeleton Creation

Dista

nce

B

ase

d

Dire

cted

Acy

clic Gra

ph


• (cont.)(cont.)Shape Graph MatchingShape Graph Matching

Two factors determine whether two nodes of the Two factors determine whether two nodes of the trees get matched: the first is a measure of the trees get matched: the first is a measure of the topological similarity of the subtrees rooted at the topological similarity of the subtrees rooted at the nodes, while the second is a measure of the local nodes, while the second is a measure of the local shape information at that node.shape information at that node.


• (Cont.)(Cont.)

But this method has a high computational cost But this method has a high computational cost and is sensitive to noise and small undulations.and is sensitive to noise and small undulations.


• Hilaga, M., Shinaagagawa, Y., Kohmura, T., and Kunii, T. L., Hilaga, M., Shinaagagawa, Y., Kohmura, T., and Kunii, T. L., Topology Matching for Fully Automatic Similarity Estimation of 3D Topology Matching for Fully Automatic Similarity Estimation of 3D ShapesShapes. In Proceedings of SIGGRAPH 2001. Computer Graphics . In Proceedings of SIGGRAPH 2001. Computer Graphics Proceedings, Annual Conference Series, 203–212, 2001.Proceedings, Annual Conference Series, 203–212, 2001.

• Multi-resolution Reeb graph is used as a search key that represents the features of a 3D shape. Torus and its Reeb

graph using a height function


• Generally, a simple reeb graph is created by using height function– μ - height of the point V: μ(V(x,y,z))=z


• The basic idea of the MRG is to develop a series The basic idea of the MRG is to develop a series of Reeb graphs for an object at various levels of of Reeb graphs for an object at various levels of detail.detail.


• The building process of MRG:The building process of MRG:

•Subdivision

–Interpolate the position of two relevant vertices in the same proportion as their value of µn(v)



•Calculate T-sets •Connect R-nodes



•Construct MRG– fine-to-coarse (reverse)


• Remaining Issues:Remaining Issues:

The structure of an MRG is sensitive to the placement of the region boundaries.

The algorithm does not distinguish between left and right.


• Actually, Height function is not appropriate Actually, Height function is not appropriate – not invariant to transformations.not invariant to transformations.

• Instead, a geodesic distance is used:Instead, a geodesic distance is used:• Not invariant to scale:Not invariant to scale:

– Normalize [0,1]:Normalize [0,1]:


• http://shape.cs.princeton.eduhttp://shape.cs.princeton.edu

Related Works: Related Works: Some Prototype Some Prototype SystemSystem

• http://shape.cs.princeton.eduhttp://shape.cs.princeton.edu


• (cont.) this system contains three parts:(cont.) this system contains three parts:


• (cont.) Part I: Acquisition(cont.) Part I: Acquisition


• (cont.) Part II: Analysis(cont.) Part II: Analysis


• (cont.) Part III: Matching(cont.) Part III: Matching


• Indexing and Retrieval of 3D Models Aided By Active Indexing and Retrieval of 3D Models Aided By Active LearningLearning, , Carnegie Mellon University, Cha Zhang and Carnegie Mellon University, Cha Zhang and Tsuhan Chen.Tsuhan Chen.


• 3D Model Retrieval Using Geodesic Distance, Taiwan 3D Model Retrieval Using Geodesic Distance, Taiwan University University

Related Works: Related Works: Other Research Other Research InstitutesInstitutes

• Germany: University of Leipzig, Germany: University of Leipzig, • http://www.informatik.uni-leipzig.de/~vranic/publishing.htm;http://www.informatik.uni-leipzig.de/~vranic/publishing.htm;• USA: CMU, USA: CMU,

http://amp.ece.cmu.edu/projects/3DModelRetrieval/ ;http://amp.ece.cmu.edu/projects/3DModelRetrieval/ ;• Greece: http://3d-search.iti.gr/ Greece: http://3d-search.iti.gr/ • USA: UC Berkeley, USA: UC Berkeley,

http://http.cs.berkeley.edu/projects/vision/shape/ http://http.cs.berkeley.edu/projects/vision/shape/ • USA: University of Texas at Austin, USA: University of Texas at Austin,

http://www.cs.utexas.edu/users/amenta/powercrust/ http://www.cs.utexas.edu/users/amenta/powercrust/ • USA: Brown University, USA: Brown University,

http://www.lems.brown.edu/vision/researchAreas/3DRecog/ohttp://www.lems.brown.edu/vision/researchAreas/3DRecog/overview.html verview.html

Our Research ObjectiveOur Research Objective

(…Omitted for the sake of the agreement (…Omitted for the sake of the agreement with FRDC (Fujitsu Research & Develop with FRDC (Fujitsu Research & Develop Center Co., Ltd.) )Center Co., Ltd.) )


(…Omitted for the sake of the agreement with (…Omitted for the sake of the agreement with FRDC (Fujitsu Research & Develop Center FRDC (Fujitsu Research & Develop Center Co., Ltd.) )Co., Ltd.) )

Outline of Our MethodsOutline of Our Methods

• Method I:Method I:



• (cont.)Method I:(cont.)Method I:



• Method II:Method II:



• (cont.) Method II:(cont.) Method II:



• Method III:Method III:


OrganizationOrganization

• Persons Involved:Persons Involved:

(1) Pu Jiantao:(1) Pu Jiantao:

(2) Xin Guyu:(2) Xin Guyu:

(3) Zhou Yu:(3) Zhou Yu:

(4) Dong Xuezhi: (4) Dong Xuezhi:

(5) Zhang Yan:(5) Zhang Yan:

SummarySummary

(1) Definition and background of 3D model (1) Definition and background of 3D model retrieval are introduced. retrieval are introduced.

(2) The reasons why the problem is difficult (2) The reasons why the problem is difficult are listed. are listed.

(3) In the section of related works, some (3) In the section of related works, some typical methods are presented.typical methods are presented.

(4) Some ideas that we have on this problem (4) Some ideas that we have on this problem are given out.are given out.

Q & AQ & A

Do anyone have some questions or Do anyone have some questions or good advices?good advices?

research on 3d model retrieval pu jiantao. [email protected] october, 2003. department of...

Documents