choi jihyun ndc2011

PRT(Precomputed Radiance Transfer) 및SH(Spherical Harmonics) 개괄

NEXON

CSO팀 클라이언트 파트

최지현

NEXON카운터 스트라이크 온라인 팀

클라이언트 프로그래밍 파트장

최지현

발표자 소개

Contents

01.Precomputed Radiance Transfer

02.Irradiance Environment Map

03.Spherical Harmonics Projection & Reconstruction

04.Spherical Harmonics “USEFUL” properties

05.Spherical Harmonics Example

06.QnA

오늘 설명할 내용

1.모든 정점(맵)에각 정점의 환경맵을 저장하자Precomputed Radiance Transfer

오늘 설명할 내용

2.환경맵을 압축해서 저장하자Spherical Harmonics as a Compression Tool

2.1.압축된 환경맵을 렌더링에 이용하자

오늘 설명할 내용

3.Then, in Detail...그럼, SH는 어떻게 사용하나?Spherical Harmonics USEFUL Property

STEP1

1.모든 정점(맵)에각 정점의 환경맵을 저장하자Precomputed Radiance Transfer

Precomputed Radiance Transfer

Precomputed Radiance Transfer

Direct Lighting

Self Shadow

Precomputed Radiance Transfer

Direct Lighting

Interreflection

Self Shadow

Precomputed Radiance Transfer

Direct Lighting

Interreflection

Subsurface Scattering

Self Shadow

Precomputed Radiance Transfer

Precomputed Radiance Transfer

Precomputed Radiance Transfer

Precomputed Radiance Transfer

STEP2

2.환경맵을 압축해서 저장하자Spherical Harmonics as a Compression Tool

2.1.압축된 환경맵을 렌더링에 이용하자

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

Irradiance Environment Map

Environment Map 을

{θ Env(Green)} 의 함수로 만들 수 있다!

)()( greenirclei fCL

Environment Map 을

{(θ, φ) Env(R,G,B)} 의 함수로 만들 수 있다!

),()( rgbpherei fSL

+ r, b channel

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

f

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

f~

0 10 30 45 60 70 80 90 100 120 135 150 160 170 180

R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255

G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255

B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

Irradiance Environment Map

f~~

Irradiance Environment Map

Environment Map 을

{(θ, φ) Env(R,G,B)} 의 함수로 만들 수 있다!

),()( rgbpherei fSL

+ r, b channel

Environment Map Env(R,G,B) 함수를

여기에 근사하는 함수로 만들 수 있다!

),(~~

),( rgbrgb ff

Approximation

Irradiance Environment Map

9x3

Low Pass Filtering

64x64x8x3

= Compressing

= Low Frequency Irradiance

STEP3

2.환경맵을 압축해서 저장하자

as a Compression Tool

2.1.압축된 환경맵을 렌더링에 이용하자

Spherical Harmonics Projection & Reconstruction

Projection To Basis

),( iL

0

0C

1

1

C

0

1C

1

1C

Spherical Harmonics Projection & Reconstruction

Environment Map Env(R,G,B) 함수를

여기에 근사하는 함수로 만들 수 있다!

),(~~

),( rgbrgb ff

Approximation

Approximation

Environment Map을 근사하는 함수를

몇개의 상수로 표현할 수 있다!

Tn

nrgb ccf 1

1

0

0 ...),(~~

Spherical Harmonics Projection & Reconstruction

Reconstruction By Basis

0

0C

1

1

C

0

1C

1

1C

),( iL

일반적으로, 앞서 언급된Spherical Coordinate 상의 Basis 를

라고 함.

),( m

ly

Spherical Harmonics Projection & Reconstruction

0 1 2 3 4-1-2-3-4m =

orderlnbandlmlliyy i

m

l :1:)1(),(),(

Spherical Harmonics Projection & Reconstruction

Spherical Harmonics Math

0),(cos

0),(cos)sin(2

0),(cos)cos(2

),(00 mPK

mPmK

mPmK

y

ll

m

l

m

l

m

l

m

l

m

l

)()12()(

)1(!)!12()1()(

)()11()()12()()1(

1

2/2

2

xPmxxP

xmxP

xPmxPlxxPm

m

m

m

m

mmm

m

m

l

m

l

m

l

)!1(

)!1(

4

)12(

m

mlK m

l

lmlNl ,

Spherical Harmonics Definition

Associated Legendre Polynomials Recursive Formulae Spherical Harmonics Normalization Constants

이해하는 사람은 천재...

Spherical Harmonics Projection & Reconstruction

n

n

nc

c

c

c

1

1

0

1

1

1

0

0

...

...

Projection To

Spherical Harmonics Basis

Function ),( m

ly

),( iL

Reconstruction By

Spherical Harmonics Basis

Function ),( m

ly

n infinite

),( iL

Spherical Harmonics Projection & Reconstruction

3

2

2

0

1

1

1

0

0

...

...

n

c

c

c

c

Projection To

Spherical Harmonics Basis

Function ),( m

ly

),( iL

Reconstruction By

Spherical Harmonics Basis

Function ),( m

ly

n=3 order

3^3=9 coefficient

),(

~

iL

MATH TIME!한방에 이해하는 사람은 천재!

나중에 집에서 천천히 생각해보세용~

Spherical Harmonics Projection & Reconstruction

Projection Math = I WANT

s

m

l

m

l dssysfc )()(

ddyfc ii sin),(),(

2

0 0

N

j

jiji xyxfN

c1

)()(4

mllidssysfcs

ii )1()()(

// Monte Carlo Integration

// Spherical Coordinate

// band index

// 모든 방향의 Environment Map

Pixel 값과 해당 방향의 SH Basis

Function 를 곱해서 다~ 더하면 됨

m

lc

Spherical Harmonics Projection & Reconstruction

Reconstruction Math = I WANT ),(

~

iL

),(),(1

0

~

n

l

l

lm

m

l

m

l yciL

),(: iL 0

0C

1

1C 0

1C 1

1C

..........

),(0

0 y

),(1

1 y ),(0

1 y ),(1

1 y

),(2

2 y ),(1

2 y ),(0

2 y ),(1

2 y ),(2

2 y2

2C 1

2C 0

2C 1

2C 2

2C

),(: iL 0

0C

1

1C 0

1C 1

1C

2

2C 1

2C 0

2C 1

2C 2

2C

..........

Spherical Harmonics Projection & Reconstruction

Reconstruction Math = I WANT ),(

~

iL

),(),(1

0

~

n

l

l

lm

m

l

m

l yciL

),(: iL 0

0C

1

1C 0

1C 1

1C

..........

),(0

0 y

),(1

1 y ),(0

1 y ),(1

1 y

),(2

2 y ),(1

2 y ),(0

2 y ),(1

2 y ),(2

2 y2

2C 1

2C 0

2C 1

2C 2

2C

STEP4

[2부]

STEP3

시간 상 설명은핵심적인 것만!

Spherical Harmonics

“USEFUL” properties

01.Quadratric Polynorminal Form

02.Double(Dot) Product

03.SH Rotation (ZYZ, Ivanic)

04.Zonal Harmonics Rotation

05.Analytic Light Source

06.Extract Dominant Direction Light

01.Quadratric Polynorminal Form

을 사용하자!)(nym

l

),( m

ly )),,(( zyxnym

l

cos

sinsin

cossin

z

y

x

),( m

ly 말고

01.Quadratric Polynorminal Form

0),(cos

0),(cos)sin(2

0),(cos)cos(2

),(00 mPK

mPmK

mPmK

y

ll

m

l

m

l

m

l

m

l

m

l

Reconstruction 을 하더라도 는 계산량이 많고Shader 로 표현하기 어려우며,

게임에서 Spherical Coordinate 는 잘 사용하지도 않는다.

),( m

ly

)()12()(

)1(!)!12()1()(

)()11()()12()()1(

1

2/2

2

xPmxxP

xmxP

xPmxPlxxPm

m

m

m

m

mmm

m

m

l

m

l

m

l

)!1(

)!1(

4

)12(

m

mlK m

l

Spherical Coodinate (θ, φ) 를 Cartesian Coordinate (x, y, z)

로 변환해서 를 직접 풀어 이용하자.),( m

ly

cos

sinsin

cossin

z

y

x

01.Quadratic Polynorminal Form

주어짂 normal 에 대하여주어짂 SH Coefficient Vector 의 SH Reconstruction 은?

)),,(( zyxnym

l

4

)(15)(,

2

15)(

,4

)13(5)(,

2

15)(,

2

15)(

,2

3)(,

2

3)(,

2

3)(

,2

1)(

222

2

1

2

20

2

1

2

2

2

1

1

0

1

1

1

0

0

yxny

xzny

zny

yzny

yxny

xny

zny

yny

ny

)()()()()(

)()()(

)()),,((

2

2

2

2

1

2

1

2

0

2

0

2

1

2

1

2

2

2

2

2

1

1

1

1

0

1

0

1

1

1

1

1

0

0

0

0

nycnycnycnycnyc

nycnycnyc

nyczyxnLi

02.Double(Dot) Product

두 함수의 곱을 Sphere 상에서적분하는 Rendering Equation 을

빠르게 계산할 수 있다!

iiiri

S

rrr dxxVxxGxLxfxL ),(),(),(),(),(

ir Lf iLG VLior or or ...

02.Double(Dot) Product

s

n

i

iidcdssVsL

2)1(

0

)()(

X =

iCsL )( idsV )( cd

o

Tn

n

n

n Lddcc

1

1

0

0

1

1

0

0 ......

1x9 9x1 1x1

02.Double(Dot) Product – PRT Revisted.

2

2

1

2

1

1

0

0

2

21

2

21

2

22

0

02

2

21

0

01

2

20

0

00

1

2

1

0

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

i

i

i

i

cc

cc

cc

cc

vc

vc

vc

vc

nn

nnn

n

"

"

"":

"":

...

...

...

"":

"":

11

22

11

00

Basis

Some

To

Light

Env

SomeBasisToEnvMapv

SomeBasisToEnvMapv

SomeBasisToEnvMapv

SomeBasisToEnvMapv

nn

nn

each vertex color = dot(envmap@vertex.to.some.basis, envlight.to.some.basis)

“Some Basis” Spherical Harmonics

“Some Basis” Ambient Cube Basis

“Some Basis” Wavelet Basis

LV

B

03.SH Rotation

SH Rotation 은마치 Vector3 의 Rotation 처럼

Linear Matrix 로표현 가능하다!

03.Rotation - ZYZ

SH – Rotation is Linear Operation!

2

2

1

2

0

2

1

2

2

2

1

1

0

1

1

1

0

0

c

c

c

c

c

c

c

c

c

2

2

1

2

0

2

1

2

2

2

1

1

0

1

1

1

0

0

b

b

b

b

b

b

b

b

b

= x

Spherical Coordinate 의 특성상 Z 축을 중심으로φ 만큼 회전하는 matrix 는 만들기 쉽다

ZXZXZZYZR 9090),,(

03.Rotation - Ivanic

ZYZ는 별로 nice 하지 않는데?

Spherical Harmonics 의 Rotation의 Recurrence Relation 한성질을 이용해서 한방에(Fast) Matrix 를 구하자

= Rotation Matrices for Real Spherical Harmonics by Ivanic

zzzyzx

yzyyyx

xzxyxx

RRR

RRR

RRR Rotation Matrix

By Local Frame

03.Rotation Invariant

Rotate Project

Rotate

Equal.BMP

03.Rotation – Ivanic - Shader

04.Zonal Harmonics

Z축에 symmetric 한

는

Rotation 을 더 빠르게 할 수 있다!

04.Zonal Harmonics

Circular Symmetry around Z-axis

Spherical Harmonic Subset

Simple & Cheap!

04.Zonal Harmonics - Rotation

xxxxx

xxxxx

xxxxx

xxxxx

xxxxx

xxx

xxx

xxx

0000

0000

0000

0000

0000

000000

000000

000000

000000001

2

2

1

2

0

2

1

2

2

2

1

1

0

1

1

1

0

0

c

c

c

c

c

c

c

c

c

xxxxx

xxxxx

xxxxx

xxxxx

xxxxx

xxx

xxx

xxx

0000

0000

0000

0000

0000

000000

000000

000000

000000001

0

0

0

0

0

0

0

2

0

1

0

0

c

c

c

04.Zonal Harmonics

1개 : 9x3=27 float

18개 : 9x3x18=486 float

모든 Vertex(Lumel) 에 대한 SH-Rotation 이 필요한 경우,

계산량이 많으므로 이경우 Zonal Harmonics 를 이용한다!

05.Analytic Light Source

우리는 Point(Direction)Light 에 익숙하니이런 형태의 정형화된 Light 로부터

SH Coefficient 를 빠르게 계산 가능하다면다양한 활용성이 기대되지 않을까?

05.Analytic Light Source

우리는 임의의 Incident Light Source : Environment Map 을SH Basis 에 Projection 해서 를 얻어낼 수 있다.m

lc

그러면, 임의의 Environment Map 이 아니라 “자주 사용하는” 정형화된형태의 Light Source 로부터 를 얻어낼 수도 있지않을까?m

lc

Cone Light Analytic Models

ddyfc ii sin),(),(

2

0 0

ddyc ii sincos),(

2

0 0

05.Analytic Light Source

0

0

)1)))(cos(cos(1)(cos(52

1

0

0

0

)sin(32

1

0

))cos(1(

2

2

1

2

0

2

1

2

2

2

1

1

20

1

1

1

0

0

c

c

c

c

c

c

c

c

c

Zonal Harmonics!

Cone Light Analytic Models

0-band

1-band

2-band

06.Extract Dominant Directional Light

주어진 SH Coefficient 에서

를 찾아내서다양한 곳에 사용해보자.

06.Extract Dominant Directional Light

0,))((1

0

2

EdcyLEn

i

ii

1

0

21

0

)(/))((n

i

i

n

i

ii dydyLc

// least square method

// minimizing E

// directional light color

)...(. 0

1

1

1

1

1 xyzcxyzcxyzcnormalizexyzdir // direction vector

// using SH linear term

// using grayscale intens.

Shader 가 지원안되는 저사양 그래픽 카드에서 사용해보자PRT Lighting 시 Specular BRDF Lighting 에 사용해보자......등등

STEP4

EXAMPLE TIME!

SH – Applications

Halo3

Crysis2

Injecting VPL to 3D Texture :

Cone Light Source (Analytic Model)

Zonal Harmonics Rotation

Spherical Harmonics Lightmap

Diffuse : Quadradtic Polynormial Form

(SH Irradiance)

Specular :

SH Rotation(Local Frame),

Double Product(Light * BRDF),

Extract Dominant Directional Light

SH - Sample

SH - Sample

SH - Sample

Diffuse Shadowed (A)

Diffuse Interreflected (RGB)

Diffuse Shadow-Interreflected (NOT-USED)

+

SphereMap NormalMap DiffuseMap

END!

Q&A

Reference

1. An Efficient Representation for Irradiance Environment Maps - Ravi Ramamoorthi, Pat Hanrahan

2. Spherical Harmonics Lighting : The Gritty Details – Robin Green

3. Stupid Spherical Harmonics (SH) Tricks - Peter-Pike Sloan

4. Lighting and Material of Halo3 – Hao Chen, Xinguo Liu

5. Light Propagation Volumes in CryEngine 3 - Anton Kaplanyan

......and more!