1

Computer Vision

一目 了然一目 了然 一一看 看 便便知知 眼睛 頭腦眼睛 頭腦Computer = Image + ArtificialComputer = Image + Artificial Vision Processing IntelligenceVision Processing Intelligence

2

Chapter 1: Cameras ○ Human Eye

Field Of View (FOV)

Width × Height

= 160 deg × 135 deg

3

• Eyeball

• Camera

4

Retina is composed of photoreceptors

Two types of photoreceptors: rods and cones

• Retina

5

(red), (green), (blue)

Rods are sensitive to intensity, motion Cones are sensitive to color, structure

Three types of cones:

6

• CCD camera

CCD (Charge-Coupled Device): rectangular grid

of electron collection site laid over a silicon wafer

CCD Image plane:

7

• Color camera

Color image plane: successive rows or columns are

made sensitive to R, G or B light using a filter

that blocks the complementary light.

Bayer pattern: a filter pattern of 2 by 2 blocks,

each of which is formed by 2 G, 1 R, and

1 B receptors.

Color image plane

8

○ Visual Sensors Animal eyes: birds, bats, snakes, fishes,

insects (fly, bee, locust, grasshopper, cricket, cicada)

Cameras: fish-eye, panoramic, omni, PTZ

Imaging Devices : telescopes, microscopes

9

○ Imaging Surfaces

Planar, Spherical, Cylindrical

○ Signals

Single value (gray images)

A few values (color images)

Many values (multi-spectral images)

10

Pinhole Cameras

Pinhole imaging model

1.1. Pinhole Cameras and Model

11

○ Big pinhole - Averaging rays blurs image Small pinhole - Diffraction effect blurs image

2 mm 1 mm 0.6 mm

0.35 mm 0.15 mm 0.07 mm

12

＊ In general, images acquired by pinhole cameras

are relatively dark because a very small set of

rays from a particular point hits the screen

＊ Pinholes Lenses

Lenses: gather light, sharp focus

Diffraction (light wavelength > hole size)

13

Perspective Projection Equations

1.1.1. Perspective Projection

14

Property:

(1) the apparent size of objects depends on

their distances from the pinhole

15

(2) The projections of two parallel lines lying in

a plane converge on a horizontal line formed

by the intersection of the image plane with

the plane parallel to and passing through the

pinhole.

16

Assume a coordinatesystem x-y-z and

17

18

19

20

(B) Algebraic method

(1) Define a) Camera coordinate system

b) Image coordinate system

(2) Prove by the projective projection equations and the limit theory

How about if the image plane and the floor plane arenot perpendicular to each other?(Assignment 1)

21

1.1.2. Affine Projection○ Three Models:

(a) Weak-perspective projection – when the

scene relief is small relative to the average

distance (z0) from the camera

22

(b) Orthographic Projection – when the scene is

far away from the camera, i.e., remote scene

(c) Para-Perspective Projection (see Ch. 2)

23

1.2. Camera with Lenses

○ Reasons for equipping lenses: a) gather light, b) sharp focus

Pinhole Cameras

Modern Cameras

24

• The laws of geometric optics

(i) Light travels in straight lines in homogeneous

media.

(ii) Reflection: (iii) Refraction:

1 2n n

○ Snell’s law

25

Proof: Find a path of light traveling from A to B with

the minimal time (Fermat’s Principle) 2 2AC b x 2 2( )BC a d x

222 2

1 2 1 2

AC BC a d xb xt

s s s s

2 2 221 2

0dt x d x

dx s b x s a d x

1 2 1 1

1 2 2 2

sin sin sin0

sin

s

s s s

Show 1 2

2 1

sin

sin

n

n

26

1.2.1. Paraxial Geometric Optics

-- Consider light rays close to the optical axis

1 1 2 2 are all small.1 1 2 2, , ,

1 1 11

1 1sin tan ( )h

R d

2 2 22

1 1sin tan ( )h

R d

27

Substituting into Snell’s law, 1 1 2 2sin sinn n

1 21 2

1 1 1 1( ) ( ),n h n hR d R d

Approximation: 1 1 2 2n n

1 1 2 2

1 2

n n n n

R d R d

Paraxial refraction equation

1 2 2 1

1 2

n n n n

d d R

28

3 5 71 1 1sin

3! 5! 7!x x x x x

Taylor expansions

2 2 2 13 5 2 (2 1)2tan

3 15 (2 )!

n n nnB xx x

x xn

Approximation: 31sin ,

3!x x x

3

tan3

xx x

1 1 2 2sin tan , sin tan 2 2

3 31 3 3

1 1 1 1

2 23 3

2 3 32 2 2 2

1 1 1 1( ) ( ) ( )

3! 3 6 3

1 1 1 1( ) ( ) ( )

3! 3 6 3

h h h h h hh

R R d d R dR d

h h h h h hh

R R d d R dR d

1 1 2 2n n Substituting into

29

Paraxial refraction equation :

2 2 21 2 2 1 1 2

1 2 1 1 2 2

1 1 1 1[ ( ) ( ) ]2 2

n n n n n nh

d d R d R d d R d

2 2 2 2

1 23 3 3 31 21 2

2 2 2 21 1 1 1 2 2 2 2

3 3 3 31 21 2

21 2 2 1 1 1 2 23 3 3 3

1 2 1 2

21 2 2 1 1 23 3 3 3

1 2 1 2

1 1 1 1( ) ( )

6 3 6 3

6 3 6 3

( )6 3 6 3

1 1 1 1( ) ( )

3 32 2

h h h hn h n h

R d R dR d R d

n n h n n h n n h n n h

R d R dR d R d

n n n n n n n nh

d d R R d R d

n n n n n nh

d d R R d R d

30

1.2.2. Thin Lenses

(a) Rays passing through O are not refracted;

(b) Rays parallel to the optical axis are

focused on the focal point F’

(c) Rays passing through the focal point F

are refracted to parallel the optical axis

31

1 2 2 1

1 2

n n n n

d d R

32

1.2.3. Thick Lenses (real lenses) -- There is a thickness between the two spherical interface surfaces.

.

33

○ Terminologies

(a) Field of view (FOV): the scene space that projects onto the image plane of the camera

FOV = , where

(b) Depth of field or Depth of focus (DOF): the range of distances within which objects are in acceptable focus.

2 1tan2

d

f

34

○ Types of aberrations(A) Spatial aberration

-- The rays from P striking the lens farther from the optical axis are focused closer to the lens -- The image of P in the image plane forms a circle of confusion (COC)

35

-- Longitudinal spherical aberration (LSA):

The distance between P’ and the intersection

of the optical axis with a ray issued from P

and refracted by the lens

-- Transverse spherical aberration (TSA):

The distance between P’ and the intersection

of the ray with the image plane

Result in shape distortion

36

(B) Chromatic aberration

-- Due to both (i) the index of refraction of a

medium and (ii) the focal length of the lens

depend on the wavelength of the incident

light rays

37

○ Compound lenses: for minimizing aberrations

○ Vignetting effect: Light beams emanating from

object points located off-axis are partially

blocked by lenses behind the aperture

38

1.4.2. Sensor Models○ The number of electrons recorded at the site (r, c) of a CCD array

where : irradiance : reflectance T : time S(r,c) : spatial domain of the cell : quantum efficiency (the number of electrons generated per unit of incident light energy)

( , )E p ( )R

( )q

( , )

( , ) ( , ) ( ) ( )p S r c

I r c T E p R q dpd

39

○ Imaging process:

Current (I) Voltage Signal Digit

CCD camera frame amplifier electronics grabber

○ The model for digital signal

( . ) ( ( , ) ( , ) ( , )

( , )) ( , )I DC BD r c N r c N r c N r c

R r c Q r c

40

: gain factor

: shot noise resulting from quantum effects

in the photo-conversion process

: dark current originating from thermal

effect

: bias introduced by the CCD electronics

R : read-out noise due to the CCD amplifier

Q : quantization noise resulting from the

digitization process

IN

DCN

BN

where

41

○ Other defects:

Blooming -- a light illuminating a site is so

bright that the charge stored at

that site overflows into adjacent

ones

Charge transfer efficiency – a source of

uncertainty

42

1.5. Note