presentasi iciast 2010

8/6/2019 Presentasi ICIAST 2010

1/40

ExtractInformationofPolarizationImagingfromLocalMatchingStereo

MohammadIqbalMohammadIqbal

OlivierMorelOlivierMorel

FabriceMeriaudeauFabriceMeriaudeau

International Conference on Intelligent & Advanced Systems 2010


2/40

Content

Background

Polarization+StereoImagingSystemResultandConclusionFutureWork

2

3

1

3

3

4


3/40

Background:WhatisPolarization? Polarization (alsopolarisation)isa

propertyofwavesthatdescr ibes theor ien ta t ion o f the i r osc i l l a t ion

Lightcanbepolarizedbyseveraldifferentprocesses Absorption Polaroidfilter Reflection Brewstersangle Scattering Lightfromthesky

Unpolarized Light

has E field at any

instant can have E

in any direction.

Polarized Light

has E field in a

certain direction

3

1


4/40

Background:WhatisPolarization? PolarizationlightInformation:

TotalIntensity AngleofPolarization:isadirectionofEfieldon

electromagneticwave/lightafterpolarized DegreeofPolarization:isaquantityusedto

describetheportionofanelectromagneticwavewhich

is

polarized.

Perfectlypolarizedwave=DOPof100%, unpolarizedwave=aDOPof0%. Partiallypolarized,=DOPsomewhereinbetween0

and100%. ApplicationsofPolarizedlight:

Polaroidsunglasses,3DMovies,Photography Polarizationmicroscopy,Liquidcrystaldisplay ComputerVision:segmentation,navigation

AoP

31


5/40

Background:WhatisStereoSystem? Stereosystemwillprovide

expandedviewsofanobject. Obtaininformationnotonlyin

2D,butalsogetthedepthofanobject(3D).

Geta

distance

information

from

twoormoreimagestakenfromdifferentpointofview(viewpoint).

scene pointscene point

optical centeroptical center

imageplane

imageplane

31


6/40

Polarization+StereoImagingImage

Acquisition

Calibration

Photometric

Calibration

Stereo

Geometric

Calibration

Extract

Polarization

Information

Remove

Outlier

Application

Feature Detection

Rectifying Image

Stereo

Matching

2


7/40

OurSystem:ImageAcquisition2


8/40

OurSystem:PhotometricCalibrationMethod of Acquisition:

1. Left

2. Right

1. LC 0

2. LC 90

2


9/40

OurSystem:PhotometricCalibration2

2

1 cos 2 sin 2 0

cos 2 cos 2 sin 2 cos 2 01 M ( )

sin 2 sin 2 cos 2 sin 2 020 0 0 0

=

Based on Mueller Matrice for Linear Polarizer

s = Mpol1.s

s = Mpol2.s = Mpol2.Mpol1.s

Target Polarizer0-180

s s s

Mpol1 Mpol2

s0

s1s

s2

s3

=

Stokes Vector :

1 Ip( ) ( s0 cos2 .s1 sin2 .s2 )

2 = + +

1. Mueller Matrix to get scalar from

2. Least Square for get a s0, s1 and s2

For =0-180 Matrix My=M.x

x=(MtM)-1Mt.y

H=(MtM)-1Mt Matrix 3x3

For =0-180s0 = s0+H(1, )*Ip()s1 = s1+H(2, )*Ip()s2 = s2+H(3, )*Ip()

Ip = a s0 + b s1 + c s2

3. Get a predicted intensity Verify with the intensity of

real Image from acquisition (Image Observe)

4. For every pixel in status alpha =0-180 :

Ipred= a s0 + b s1 + c s2

Verify = Ipred-Iobs

result 0 is ideal

result 1-2 is toleranceresult >2 is to much noise

2


10/40

OurSystem:VisualResultofPhotometricCalibration

Left Polarizer

2 pixel : A(270,497)

B(198,501)

Right Polarizer

2 pixel : A(270,497)

B(198,501)

A BLC 0 for 2 pixel : A(270,497)

B(198,501)

LC 90 for 2 pixel :

A(270,497) B(198,501)

A B

2


11/40

OurSystem:GeometricCalibrationandImageRectifying

Left Right

Calibration use aBouguet Toolbox, toget a camera :- Internal parameter- External parameter

Images rectifying use all parameter to transform geometrically

2


12/40

OurSystem:FeatureDetector,StereoMatchingandRemoveOutliers

Sparse Feature DetectorWe use Harris corner detector

method, detector based-on the imagegradient.

Harris and Stephens Method :

1. R = det(M)- ktr(M)2

2. E = min (Eigenvalue of M)

1 1 2 2

2 2u ,v

1 1 2 2

u ,v u ,v

I ( u,v ) I I ( u d ,v ) I NSSD

I ( u,v ) I I ( u d ,v ) I

+ = +

Dense Stereo Matching AlgorithmWe use Normalized SSD (Sum of Squared

Differences) because this algorithm gave bettermatching results when applied to our polarized

images, compared to other local matching

algorithms (NCC, SSD, SAD, census and rank).

((I0+I90)/2) I45

2

DetailDetail


13/40

OurSystem:DenseStereoMatchingandRemoveOutliers

Remove Outliers

We use RANSAC. it is an iterativemethod to estimate parameters ofa mathematical model from a setof observed data which containsoutliers. This algorithm was firstpublished by Fischler and Bolles in1981.

Putative Match Inlying Matches

2

DetailDetail


14/40

OurSystem:ExtractPolarizationInfoBasic technique to obtain the state of polarization of incident light is to capturethree different intensity images through a set of polarization filters. If we have I0,I45

and I90

(representation of the image intensity measurements taken at an

angle of polarizer of 0, 45and 90), we can compute : angle of polarization,degree of polarization and total intensity.

totI

I ( 1 cos( 2 2 ))2

= +

tot0

tot45

tot90

I I ( 1 cos 2 )

2

I I ( 1 sin 2 )

2

I

I ( 1 cos 2 )2

= +

=

=

45

tot

90

tot

2Isin 2 1

I

2Icos 2 1

I

=

=

45

tot

90

tot

2I1I

a tan / 22I

1I

=

45

tot

45

tot

90

tot

2I1I

2I1

Isin( a tan )

2I

1 I

=

I = Imin+ Imax

Angle ofpolarization

Degree ofpolarization

Total

Intensity

= Angle of polarizer rotation

2


15/40

OurSystem:ImageAcquisitionforVerifyPolarizationInfo

55mm

400mm

Pol1Pol2

LC

2 State Liquid CrystalPolarizer : 0, 90

5 State Incident light :0, 10, 20, 30, 45

Object

right : 45

2


16/40

OurSystem:ExtractPolarizationInfo

IncidentIncident

LightLight

angleangle

IntensityIntensity

OfPolOfPol

MeanofMeanof

AOPAOP

MeanofMeanof

DOPDOP

%ofAverage%ofAverage

ErrorofErrorof

AOPAOP

0 0.3863 2.1586 0.7470 0.021610 0.3674 8.7851 0.6139 0.187920 0.3409 16.0545 0.5019 0.360530 0.4062 26.6184 0.4561 0.566245 0.4042 39.7027 0.4437 0.8470

0and 90

45

Open Image left 0 and 90

and right 45

Get a graylevel value on

every pixel founded by

Local Matching Algorithm

Compute Angle of

Polarization (AoP)

Compute Degree ofPolarization (AoP)

Compute Intensity ofPolarization

Visualization Polarization

Based Stereo Matching

AOPAOP = Angle of PolAngle of Pol

DOPDOP = Degree of PolDegree of Pol

33


17/40

OurSystem:VizualizePolarizationInfo(x,y)

AOP /AOP /

Angle of

Polarization

DOP /DOP /

Degree of

Polarization

(x2,y2)

(x1,y1)

33

10

20

30


18/40

ResumeResult11 2233

44

55

the polarization information is

extracted from the inlier pixelsand visualized.

Each source image :

Left :(i(0)+i(90))/2 Right : i(45)

Extract Harris features

Stereo matching by NSSD

Each pixel match found in thewrong place is rejected byRANSAC method

11

22

33

44

55

33

DetailDetail


19/40

Conclusion Inthiswork,wehavedone:

Implemented

of

polarization

visionImplemented

of

polarization

vision systemin

stereo

imaging.

SetupdesignofImagingsystemSetupdesignofImagingsystem withliquidcrystalpolarizerinonesideandafixedpolarizerintheothersidethatcansensepartiallinearlypolarizedlightandcomputationallyprocesspolarizationcomponent.

VisualizeapolarizationVisualizeapolarization componentintheoutput. Experimentsshowthatourmethodexhibitsgood

performanceincomplexbackground,especiallywhenthereissomelightreflectedfromspecularparts.

33


20/40

FutureWorkImage

Acquisition

Calibration

Photometric

Calibration

Stereo

Geometric

Calibration

Extract

Polarization

Information

Remove

Outlier

Feature Detection

Rectifying Image

Stereo

Matching

Water region

Segmentation

3D

Recontruction

SelfCalibration

Improvement

Photometric Invariant

Feature Detector

4


21/40

Remerciement Thisworkwassupportedby:

BureauofPlanningandForeignCooperation,MinistryofNationalEducation,RepublicofIndonesia.

GunadarmaUniversity,IndonesiaOpticsandallresearchequipmentsarecourtesy

ofLE2IIUTLeCreusot,France.


22/40

Terima Kasih

Thank You

Question?Question?


23/40

DetailVisualResultDetailVisualResult


24/40

Scene 10 :SourceImages(Rectified)Epipolar

line

33


25/40

Scene 10 :HarrisFeature33


26/40

Scene 10 :NSSDMatching33


27/40

Scene 10 :NSSDMatchingPixelPixel CorrespondenceCorrespondence

0 100 200 300 400 500 600 7000

0.5

1

1.5

2

2.5

3

3.5

1st Pixel1st Pixel 2nd Pixel2nd Pixel LastLast PixelPixel

0 200 400 600 8000

0.5

1

1.5

2

2.5

3

0 200 400 600 8000

0.5

1

1.5

2

2.5

3

3.5

Compare to

Find the lowest(NSSD)

12..

.

n

1 2 . . . n12..

.

n

1 2 . . . n

Windows11x11

Left Right

33


28/40

Scene 10 :Remove OutliersPutative Match

Number of inliers was 110 (61%)Number of putative matches was 180

Inlying Matches

33


29/40

Scene 10 :Visualisasi Polarization33


30/40

Visualization forall Scene33


31/40

Visualization forall SceneScene 0

(Yellow Box)

Scene 10

(Green Box)

33


32/40

Visualization forall SceneScene 20(Blue Box)

Scene 30

(Red Box)

33


33/40

Visualization forall SceneScene 45(Cyan Box)

33BackBack


34/40

AppendixAppendix


35/40

HarriscornerdetectorThere is many possibility detector to get a feature point fromimages. One of well-known method is Harris corner detector.

This method based on the use of the image gradient.

The squared image derivatives

should be smoothed by convolving

them with a Gaussian filter g.2 2

x x I g I =

Defined autocorrelation matrix M

for every pixel:2

x x y

w w

2

x y y

w w

I I I

M I I I

=

where Wis a 33 neighborhood

around the point.

Compute the derivatives of the

intensity image in the x and y

directions for every pixel.

x

II

x

=

y

II

y

=

2 2

x y

I ,I

M is for derive a

measure of

cornerness

rank M = 0 : the

pixel belongs to an

homogeneous regionrank M = 1 : an edge

(significant gradients

in one directions),

rank M = 2 : a

corner (significant

gradients in both

directions).

Harris and Stephens Method :1. R = det(M)- k tr(M)2

2. E = min (Eigenvalue of M)

use of the determinant

and the trace (sum of

the diagonalelements ) of M to

detect corners

Use the smaller

eigenvalue of M for

detect a corner.

2BackBack


36/40

RANSAC(RAndom

SAmple

Consensus)

InlyingMatches

2BackBack


37/40

ComputePolarizationInfoPartial linear polarization can be measured at a pixel level

by the transmitted radiance through a polarization filter.The radiance varies sinusoidally with filter orientation. This

is based on the work of Wolff et al. :

Total Intensity

I = Imin

+ Imax

= I0

+ I90

Angle of Polarization :

= 0.5 * arctan ( ( I0 + I90 - 2I45 ) / I90 - I0If I90 < I0 [ if (I45 < I0 ) = +90

else = +90

Partial Polarization or degree of polarization :

= ( I90 - I0 ) / ( I90 - I0 ) cos 2

StepStep :: Open Image left 0 and 90and right 45

Get a graylevel value onevery pixel founded by

Local Matching Algorithm

Compute Angle of

Polarization (AoP)

Compute Degree of

Polarization (AoP)

Compute Intensity of

Polarization

Visualization Polarization

Based Stereo Matching

2


38/40

ComputePolarizationInfo1

2 2 2

1 2 3

s1arccos

2 s s s

=

+ +s0 =Itots1 =Ipol cos 2

s2 =Ipol sin 2cos

s3 =Ipol sin 2sin

3

2

sarctan

s

=

Angle of Polarization Phase of Polarization

We use a linear polarizer, so we can simplified polarization imaging without use

S3 and phase of polarization. So, Stokes vector of partially linearly polarized wave

can be written :

s0 =Itots1 = Ipol cos 2s2 = Ipol sin 2

s3 = 0

0 1 2

1 I( ) ( S S cos 2 S sin 2 )

2

= + +

totI I( ) (1 cos( 2 2 ))2

= +

= Angle of polarizer rotation

Degree of Polarization2 2 2

pol 1 2 3

tol 0

I s s s

I s

+ += =

2


39/40

ComputePolarization

Info

totI I (1 cos( 2 2 ))

2 = +

tot0

tot45

tot90

I I (1 cos 2 )

2

I I (1 sin 2 )

2I

I (1 cos 2 )2

= +

=

=

45

tot

90

tot

2Isin 2 1 ( a )I

2Icos 2 1 ( b )

I

=

=

45

tot

90

tot

2I1

Ia tan / 22I

1I

=

45

tot

45

tot

90

tot

2I1

I

2I1I

sin( a tan )2I

1I

=

2BackBack


40/40

VizualizePolarization

Info

Cosinus Law

(x,y)

x1=(x+b)

y1 = (y-c)

y2=(y+c)

x2=(x-b)

(x,y)AOP /AOP /

Angle of

Polarization

DOP /DOP /

Degree of

Polarization

(x2,y2)

(x1,y1)Length = DOP/2

b=length*cos(AOP)c=length*sin(AOP)

x1=x+b;y1=y-c

x2=x-b;y2=y+c(x,y) ((x+b),y)

((x+b),(y-c))

x(col)

y(row)

a

b = a cos

= a sin c

2BackBack

presentasi iciast 2010

Documents