21 hdr

7/30/2019 21 Hdr

1/52

High Dynamic Range Images

15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005

with a lot of slides stolen from

Paul Debevec and Yuanzhen Li,

Alyosha Efros

7/30/2019 21 Hdr

2/52

The Grandma ProblemThe Grandma Problem

7/30/2019 21 Hdr

3/52

Problem: Dynamic RangeProblem: Dynamic Range

15001500

11

25,00025,000

400,000400,000

2,000,000,0002,000,000,000

The real world is

high dynamic range.

The real world is

high dynamic range.

7/30/2019 21 Hdr

4/52

pixel (312, 284) = 42pixel (312, 284) = 42

ImageImage

42 photos?42 photos?

7/30/2019 21 Hdr

5/52

Long ExposureLong Exposure

10-6 106

10-6 106

Real world

Picture

0 to 255

High dynamic range

7/30/2019 21 Hdr

6/52

Short ExposureShort Exposure

10-6 106

10-6 106

Real world

Picture

High dynamic range

0 to 255

7/30/2019 21 Hdr

7/52

Camera CalibrationCamera Calibration

Geometric How pixel coordinates relate to directions in the

world

Photometric

How pixel values relate to radiance amounts in

the world

GeometricHow pixel coordinates relate to directions in the

world

Photometric

How pixel values relate to radiance amounts inthe world

7/30/2019 21 Hdr

8/52

The ImageAcquisition Pipeline

The ImageAcquisition Pipeline

scene

radiance(W/sr/m )

scene

radiance(W/sr/m )

sensor

irradiance

sensor

irradiance

sensor

exposure

sensor

exposure

latent

image

latent

image

LensLens ShutterShutter FilmFilm

Electronic CameraElectronic Camera

22

tt

7/30/2019 21 Hdr

9/52

film

density

film

density

analog

voltages

analog

voltages

digital

values

digital

values

pixel

values

pixel

values

DevelopmentDevelopment CCDCCD ADCADC RemappingRemapping

7/30/2019 21 Hdr

10/52

loglog Exposure =Exposure =loglog (Radiance(Radiance ** tt))

Imaging system response functionImaging system response function

PixelPixel

valuevalue

0

255

(CCD photon count)

7/30/2019 21 Hdr

11/52

Varying ExposureVarying Exposure

7/30/2019 21 Hdr

12/52

Camera is not a photometer!Camera is not a photometer!

Limited dynamic range Perhaps use multiple exposures?

Unknown, nonlinear response

Not possible to convert pixel values toradiance

Solution: Recover response curve from multiple

exposures, then reconstruct theradiance map

Limited dynamic range Perhaps use multiple exposures?

Unknown, nonlinear response

Not possible to convert pixel values toradiance

Solution:Recover response curve from multiple

exposures, then reconstruct theradiance map

7/30/2019 21 Hdr

13/52

Recovering High Dynamic RangeRadiance Maps from PhotographsRecovering High Dynamic RangeRadiance Maps from Photographs

Paul Debevec

Jitendra Malik

Paul Debevec

Jitendra Malik

August 1997August 1997

Computer Science Division

University of California at Berkeley

Computer Science Division

University of California at Berkeley

7/30/2019 21 Hdr

14/52

Ways to vary exposureWays to vary exposure Shutter Speed (*)

F/stop (aperture, iris)

Neutral Density (ND) Filters

Shutter Speed (*)

F/stop (aperture, iris)

Neutral Density (ND) Filters

7/30/2019 21 Hdr

15/52

Shutter SpeedShutter Speed

Ranges: Canon D30: 30 to 1/4,000 sec.

Sony VX2000: to 1/10,000 sec. Pros:

Directly varies the exposure

Usually accurate and repeatable

Issues:

Noise in long exposures

Ranges: Canon D30: 30 to 1/4,000 sec.

Sony VX2000: to 1/10,000 sec. Pros:

Directly varies the exposure

Usually accurate and repeatable

Issues:

Noise in long exposures

7/30/2019 21 Hdr

16/52

Shutter SpeedShutter Speed

Note: shutter times usually obey a power

series each stop is a factor of 2

, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000

sec

Usually really is:

, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024

sec

Note: shutter times usually obey a power

series each stop is a factor of 2

, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000

sec

Usually really is:

, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024

sec

7/30/2019 21 Hdr

17/52

3

33

111

2

22

tt ==11 secsec

3

33

111

2

22

tt ==1/161/16 secsec

3

33

1 11

2 22

tt ==44 secsec

3

33

111

2

22

tt ==1/641/64 secsec

The AlgorithmThe Algorithm

Image seriesImage seriesImage series

3

33

111

2

22

tt ==1/41/4 secsec

Exposure = Radiance tExposure = Radiance t

log Exposure = log Radiance + log tlog Exposure = log Radiance + log t

Pixel Value Z = f(Exposure)Pixel Value Z = f(Exposure)

7/30/2019 21 Hdr

18/52

Response CurveResponse Curve

ln Exposureln Exposure

Assuming unit radiancefor each pixel

Assuming unit radiancefor each pixel

Pixelvalue

Pixelvalue

Afteradjusting radiances toobtain a smooth response

Afteradjusting radiances toobtain a smooth response

curvecurve

333

111

222

ln Exposureln Exposure

Pixelvalue

Pixelvalue

7/30/2019 21 Hdr

19/52

The MathThe Math

Let g(z)be the discrete inverse response function

For each pixel site i in each imagej, want:

Solve the overdetermined linear system:

Let g(z)be the discrete inverse response function

For each pixel site i in each imagej, want:

Solve the overdetermined linear system:

fitting term smoothness term

lnRadiancei + lntj g(Zij )[ ]2

j=1

P

i=1

N

+ g (z)2z=Zmin

Zmax

lnRadiancei +lntj = g(Zij )

7/30/2019 21 Hdr

20/52

MatlabCode

MatlabCode

f unct i on [ g, l E] =gsol ve( Z, B, l , w)

n = 256;A = zer os( si ze( Z, 1) *si ze( Z, 2) +n+1, n+si ze( Z, 1) ) ;b = zer os( si ze( A, 1) , 1) ;

k = 1; %% I ncl ude t he dat a- f i t t i ng equat i onsf or i =1: si ze( Z, 1)

f or j =1: si ze( Z, 2)wi j = w( Z( i , j ) +1) ;A( k, Z( i , j ) +1) = wi j ; A( k, n+i ) = - wi j ; b( k, 1) = wi j * B( i , j ) ;k=k+1;

endend

A( k, 129) = 1; %% Fi x t he cur ve by set t i ng i t s mi ddl e val ue tk=k+1;

f or i =1: n- 2 %% I ncl ude t he smoot hness equat i onsA( k, i ) =l *w( i +1) ; A( k, i +1) =- 2*l *w( i +1) ; A( k, i +2) =l *w( i +1) ;k=k+1;

end

x = A\ b; %% Sol ve t he syst em usi ng SVD

g = x( 1: n) ;l E = x( n+1: s i ze( x, 1) ) ;

7/30/2019 21 Hdr

21/52

Results: Digital CameraResults: Digital Camera

Recovered responseRecovered response

curvecurve

log Exposurelog Exposure

P

ixelvalu

e

P

ixelvalu

e

Kodak DCS460

1/30 to 30 sec

7/30/2019 21 Hdr

22/52

Reconstructed radiance mapReconstructed radiance map

7/30/2019 21 Hdr

23/52

Results: Color FilmResults: Color Film

Kodak Gold ASA 100, PhotoCD Kodak Gold ASA 100, PhotoCD

7/30/2019 21 Hdr

24/52

Recovered Response CurvesRecovered Response Curves

Red Green

RGBBlue

7/30/2019 21 Hdr

25/52

The

Radiance

Map

The

Radiance

Map

7/30/2019 21 Hdr

26/52

The

Radiance

Map

The

Radiance

Map

Linearly scaled toLinearly scaled to

display devicedisplay device

7/30/2019 21 Hdr

27/52

Portable FloatMap (.pfm)Portable FloatMap (.pfm) 12 bytes per pixel, 4 for each channel

sign exponent mantissa

PF

768 512

1

Floating Point TIFF similarFloating Point TIFF similar

Text header similar to Jeff Poskanzers .ppmimage format:

7/30/2019 21 Hdr

28/52

(145, 215, 87, 149) =

(145, 215, 87) * 2^(149-128) =(1190000, 1760000, 713000)

(145, 215, 87, 149) =

(145, 215, 87) * 2^(149-128) =(1190000, 1760000, 713000)

(145, 215, 87, 103) =

(145, 215, 87) * 2^(103-128) =(0.00000432, 0.00000641, 0.00000259)

(145, 215, 87, 103) =

(145, 215, 87) * 2^(103-128) =(0.00000432, 0.00000641, 0.00000259)

Red Green Blue ExponentRed Green Blue Exponent

32 bits / pixel32 bits / pixel

Ward, Greg. "Real Pixels," in Graphics Gems IV, edited by James Arvo, Academic Press, 1994

Radiance Format

(.pic, .hdr)

Radiance Format

(.pic, .hdr)

7/30/2019 21 Hdr

29/52

ILMs OpenEXR (.exr)ILMs OpenEXR (.exr) 6 bytes per pixel, 2 for each channel, compressed

sign exponent mantissa

Several lossless compression options, 2:1 typical

Compatible with the half datatype in NVidia's Cg Supported natively on GeForce FX and Quadro FX

Available at http://www.openexr.net/
http://www.openexr.net/http://www.openexr.net/

7/30/2019 21 Hdr

30/52

NowWhat?NowWhat?

7/30/2019 21 Hdr

31/52

Tone MappingTone Mapping

10-6 106

10-6 106

Real WorldRay Traced

World (Radiance)

Display/

Printer0 to 255

High dynamic range

How can we do this?Linear scaling?, thresholding? Suggestions? How can we do this?Linear scaling?, thresholding? Suggestions?

7/30/2019 21 Hdr

32/52

Simple Global OperatorSimple Global Operator

Compression curve needs to

Bring everything within range

Leave dark areas alone

In other words

Asymptote at 255

Derivative of 1 at 0

Compression curve needs to

Bring everything within range

Leave dark areas alone

In other words

Asymptote at 255

Derivative of 1 at 0

7/30/2019 21 Hdr

33/52

Global Operator (Reinhart et al)Global Operator (Reinhart et al)

world

worlddisplayL

LL += 1

7/30/2019 21 Hdr

34/52

Global Operator ResultsGlobal Operator Results

7/30/2019 21 Hdr

35/52

DarkestDarkest 0.1%0.1% scaledscaledto display deviceto display device

Reinhart OperatorReinhart Operator

7/30/2019 21 Hdr

36/52

What do we see?What do we see?

Vs.

7/30/2019 21 Hdr

37/52

What does the eye sees?What does the eye sees?

The eye has a huge dynamic rangeDo we see a true radiance map?

7/30/2019 21 Hdr

38/52

MetamoresMetamores

Can we use this for range compression?

7/30/2019 21 Hdr

39/52

Compressing Dynamic RangeCompressing Dynamic Range

range

range

This reminds you of anything?

7/30/2019 21 Hdr

40/52

Compressing and CompandingHigh Dynamic Range Images with

Subband Architectures

Compressing and CompandingHigh Dynamic Range Images with

Subband Architectures

Yuanzhen Li, Lavanya Sharan,

Edward Adelson

Massachusetts Institute of Technology

7/30/2019 21 Hdr

41/52

Dynamic Range ProblemDynamic Range Problem

Source: Shree Nayar

7/30/2019 21 Hdr

42/52

Range CompressionRange Compression

Method: Gamma or log on intensities.

Problem: loss of detail.

Solution: filtering.Problem: halos.

+

+

compresslowpass

highpass

Halos!!

7/30/2019 21 Hdr

43/52

Multiscale Subband DecompositionMultiscale Subband Decomposition

spatial frequency

orientation

lowpass

residue

Choice of filters:Wavelets, QMFs, Laplacian, etc.

They all worked.

7/30/2019 21 Hdr

44/52

Point Nonlinearity on SubbandsPoint Nonlinearity on Subbands

point nonlinearitylimits range

Original subband Modified subband

Problem: Nonlinear distortion.

flattened peak

7/30/2019 21 Hdr

45/52

smooth

gain(x) = b(x) / b(x) = =

Smooth Gain ControlSmooth Gain Control

x =

Smooth Gain Control ReducesSmooth Gain Control Reduces

7/30/2019 21 Hdr

46/52

Smooth Gain Control Reduces

Distortion

Smooth Gain Control Reduces

Distortion

Smooth gain controlPoint nonlinearity

Distorted. Distortion reduced.

7/30/2019 21 Hdr

47/52

Smooth Gain Control on SubbandsSmooth Gain Control on Subbands

rectify

blur

activity map gain map

Fattal et al. 2002Ours

7/30/2019 21 Hdr

48/52

Reinhard et al. 2002 Durand & Dorsey 2002

Reinhard et al. 2002

7/30/2019 21 Hdr

49/52


7/30/2019 21 Hdr

50/52



7/30/2019 21 Hdr

51/52



7/30/2019 21 Hdr

52/52


21 hdr

Documents