transformations in ray tracing - ustcstaff.ustc.edu.cn/~lgliu/courses/computergraphics... · mit...

计算机图形学 Computer Graphics

刘利刚 [email protected]

http://staff.ustc.edu.cn/~lgliu

mailto:[email protected]

http://staff.ustc.edu.cn/~lgliu

Computer Animation

Skinning and Enveloping

The slide are from Durand from MIT.

MIT EECS 6.837 - Durand

Before getting started• One more word about rotations

2

Tuesday, October 5, 2010

MIT 6.837 - Durand

Extracting rotations• We have seen that rotation is best interpolated with axis-

angle representation or, better, quaternions• But often, we are simply given 3x3 or 4x4 matrices that

contains rotations, translations, scale, shear, etc.

• It would be useful to extract the rotation part, do the (complicated) right thing, and interpolate the rest linearly

3Tuesday, October 5, 2010

• Polar decomposition breaks arbitrary matrix M into Rotation Q (+potential reflection) Symmetric positive definite S

(anisotropic scale)

• Extract rotation part Q, interpolate it using quaternions Interpolate S using some other method.

• Shoemake, K. and Duff, T. 1992. Matrix animation and polar decomposition. In Proceedings of the Conference on Graphics interface '92, pp258-264. http://www.cs.wisc.edu/graphics/Courses/838-s2002/Papers/polar-decomp.pdf

MIT 6.837 - Durand

Non-orthonormal 3x3 matrix


http://www.cs.wisc.edu/graphics/Courses/838-s2002/Papers/polar-decomp.pdf

http://www.cs.wisc.edu/graphics/Courses/838-s2002/Papers/polar-decomp.pdf

MIT 6.837 - Durand

Polar Decomposition Algorithm• Basic idea:

rotations are the matrices such that M-1=MT

• Massage the matrix until it satisfies this


MIT 6.837 - Durand

Polar Decomposition Algorithm• Given 3x3 Matrix M• Compute the rotation factor Q by averaging the matrix with

its inverse transpose until convergence: Set Q0 = M, then Qi+1 = 1/2(Qi+ Qi–T) until Qi+1 – Qi ≈ 0.


MIT 6.837 - Durand

Questions?



Traditional Animation

ACM © 1997 “Multiperspective panoramas for cel animation”8

• Draw each frame by hand– great control, but tedious

• Reduce burden with cel animation– Layer, keyframe, inbetween, …– Example: Cel panoramas (Disney’s

Pinocchio)


http://en.wikipedia.org/wiki/Cel_animation

http://en.wikipedia.org/wiki/Cel_animation


Traditional Animation Principles• The in-betweening, was once a job for apprentice animators.

Splines accomplish these tasks automatically. However, the animator still has to draw the keyframes. This is an art form and precisely why the experienced animators were spared the in-betweening work even before automatic techniques.

• The classical paper on animation by John Lasseter from Pixar surveys some the standard animation techniques:

• "Principles of Traditional Animation Applied to 3D Computer Graphics,“ SIGGRAPH'87, pp. 35-44.

• See also

9


http://doi.acm.org/10.1145/37401.37407

http://doi.acm.org/10.1145/37401.37407

http://doi.acm.org/10.1145/37401.37407

http://doi.acm.org/10.1145/37401.37407


Example: Squash and stretch• Squash: flatten an object or character by pressure or by

its own power

• Stretch: used to increase the sense of speed and emphasize the squash by contrast

10



Example: Timing• Timing affects weight:

– Light object move quickly– Heavier objects move slower

• Timing completely changes the interpretation of the motion.

11



See also• http://www.pixar.com/howwedoit/index.html

12


http://www.pixar.com/howwedoit/index.html

http://www.pixar.com/howwedoit/index.html


Computer Animation• How do we describe and generate motion of

objects in the scene?

• Two very different contexts: – Production (offline)

• Can be hardcoded, entire sequence know beforehand– Interactive (e.g. games, simulators)

13



Computer Animation• How do we describe and generate motion of

objects in the scene?

• Two very different contexts: – Production (offline)

• Can be hardcoded, entire sequence know beforehand– Interactive (e.g. games, simulators)

• Needs to react to user interaction, sequence not known14



Plan• Types of Animation (overview)

– Keyframing– (Procedural)– Physically-based

• Animation Controls

• Character animationusing skinning/enveloping

15CERNTuesday, October 5, 2010


Types of Animation: Keyframing• Specify scene only at

some instants of time• Generate in-betweens automatically

16

t1

t2 t3

t4



Types of Animation: Keyframing

17

• Specify scene only atsome instants of time

• Generate in-betweens automatically



Types of Animation: Procedural• Describes the motion algorithmically• Express animation as a function of

small number of parameters• Example

– a clock with second, minute and hour hands– express the clock motions in terms of

a “seconds” variable• the clock is animated by

changing this variable

• Another example: Grass in the wind,tree canopies, etc.

18



Types of Animation: Physically-Based

• Assign physical properties to objects– Masses, forces, etc.

• Also procedural forces (like wind)• Simulate physics by solving equations of motion

– Rigid bodies, fluids, plastic deformation, etc.• Realistic but difficult to control

v0

m g

19



Example: Water Simulation

20

Losasso, F., Talton, J., Kwatra, N. and Fedkiw, R., "Two-way Coupled SPH and Particle Level Set Fluid Simulation", IEEE TVCG 14, 797-804 (2008).Tuesday, October 5, 2010

http://physbam.stanford.edu/~fedkiw/papers/stanford2007-05.pdf







Another Example• Physically-Based Character Animation

– Specify keyframes, solve for physically valid motion that interpolates them by “spacetime optimization”

• Anthony C. Fang and Nancy S. Pollard, 2003. Efficient Synthesis of Physically Valid Human Motion, ACM Transactions on Graphics 22(3) 417-426, Proc. SIGGRAPH 2003.– http://graphics.cs.cmu.edu/nsp/projects/spacetime/

spacetime.html

21


http://graphics.cs.cmu.edu/nsp/papers/acfSig03.pdf




http://graphics.cs.cmu.edu/nsp/projects/spacetime/spacetime.html





Because we are Lazy...• Animation is (usually) specified using some form

of low-dimensional controls as opposed to remodeling the actual geometry for each frame.

22

Can you think of examples?



• Animation is (usually) specified using some form of low-dimensional controls as opposed to remodeling the actual geometry for each frame.– Example: The joint angles (bone transformations) in a

hierarchical character determine the pose– Example: A rigid motion is represented by

changing the object-to-world transformation(rotation and translation).

23

t1 t2

t3

t4

Because we are Lazy...



• Animation is (usually) specified using some form of low-dimensional controls as opposed to remodeling the actual geometry for each frame.– Example: The joint angles (bone transformations) in a

hierarchical character determine the pose– Example: A rigid motion is represented by

changing the object-to-world transformation(rotation and translation).

24

t1 t2

t3

t4

“Blendshapes” are keyframes that are just snapshots of the entire geometry.

Because we are Lazy...



Example of Higher-Level Controls• Ken Perlin’s facial expression

applet– http://mrl.nyu.edu/~perlin/

experiments/facedemo/

• Lower-level controls are mapped to semantically meaningful higher-level ones– “Frown/smile” etc.

25


http://mrl.nyu.edu/~perlin/experiments/facedemo/





Building 3D models and their animation controls is a major component of every animation pipeline.

Building the controls is called “rigging”.

26



Articulated Character Models• Forward kinematics describes

the positions of the body parts as a function of joint angles– Body parts are

usually called “bones”– Angles are the low-

dimensional control.• Inverse kinematics specifies

constraint locations for bonesand solves for joint angles.

27



Skinning Characters• Embed a skeleton into a

detailed character mesh• Animate “bones”

– Change the jointangles over time

– Keyframing, procedural, etc.• Bind skin vertices to bones

– Animate skeleton, skin will move with it

28

Epi

c G

ames



Skinning Characters• Embed a skeleton into a

detailed character mesh• Animate “bones”

– Change the jointangles over time

– Keyframing, procedural, etc.• Bind skin vertices to bones

– Animate skeleton, skin will move with it

29

Epi

c G

ames



Motion Capture• Usually uses optical markers and multiple

high-speed cameras• Triangulate to get marker 3D position

– (Again, structure from motion and projective geometry, i.e., homogeneous coordinates)

• Captures style, subtle nuances and realism• But need ability to record someone

30



Example: Facial Motion Capture• http://ps3.ign.com/dor/objects/826461/tiger-

woods-pga-tour-07/videos/tigerucap07_080806_01.html

31


http://ps3.ign.com/dor/objects/826461/tiger-woods-pga-tour-07/videos/tigerucap07_080806_01.html







Motion Capture• Motion capture records

3D marker positions– But character is controlled

using animation controls that affect bone transformations!

• Marker positions must be translated into character controls (“retargeting”)– A kind of inverse

kinematics!32

?


MIT EECS 6.837 - Durand 33

ILM / Walt Disney Pictures



ILM / Walt Disney Pictures



Questions?

35



Skinning/Enveloping

36

Epic Games / ign.com

Gears of War 2



Skinning• We know how to animate a

bone hierarchy– Change the joint angles, i.e.,

bone transformations, over time (keyframing)

37

Epi

c G

ames



Skinning• We know how to animate a

bone hierarchy– Change the joint angles, i.e.,

bone transformations, over time (keyframing)

• Embed a skeleton into a detailed character mesh

• Bind skin vertices to bones– Animate skeleton, skin will

move with it– But how?

38

Epi

c G

ames



Skinning/Enveloping• Need to infer how skin deforms

from bone transformations.• Most popular technique:

Skeletal Subspace Deformation (SSD), or simply Skinning– Other aliases

• vertex blending• matrix palette skinning• linear blend skinning

From wikipedia 39



SSD / Skinning• Each bone has a deformation of

the space around it (rotation, translation)

40




the space around it (rotation, translation)– What if we attach each

vertex of the skin to a single bone?

41





vertex of the skin to a single bone?• Skin will be rigid, except at joints where it will

stretch badly

42





vertex of the skin to a single bone?• Skin will be rigid, except at joints where it will

stretch badly– Let’s attach a vertex to many bones at once!

• In the middle of a limb,the skin points follow the bone rotation (near-)rigidly

• At a joint, skin is deformed according to a “weighted combination” of the bones

43



Examples

44James & Twigg 2005

Colored triangles are attached to 1

bone

Black triangles are attached to more than 1

Note how they are near

joints



Examples

45

Colored triangles are attached to 1

bone

Black triangles are attached to more than 1

Note how they are near

jointsJames & Twigg 2005



Vertex Weights• We’ll assign a weight wij

for each vertex pi for each bone Bj.– “How much vertex i should move with bone j”

46




for each vertex pi for each bone Bj.– “How much vertex i should move with bone j”– wij = 1 means pi is rigidly attached to Bj.

47


• We’ll assign a weight wijfor each vertex pi for each bone Bj.– “How much vertex i should move with bone j”– wij = 1 means pi is rigidly attached to bone j.


Vertex Weights

48Wang & Phillips 2002Tuesday, October 5, 2010



for each vertex pi for each bone Bj.– “How much vertex i should move with bone j”– wij = 1 means pi is rigidly attached to bone j.

• Weight properties– Usually want weights to be non-negative

49





• Weight properties– Usually want weights to be non-negative– Also, want the sum over all bones

to be 1 for each vertex• This means translation independence.• (Again, a partition of unity – remember splines basis

functions?)50



Vertex Weights cont’d• We’ll assign a weight wij


• We’ll limit the number of bones N that can influence a single vertex– N=4 bones/vertex is a usual choice– Why?

51



Vertex Weights cont’d• We’ll assign a weight wij


• We’ll limit the number of bones N that can influence a single vertex– N=4 bones/vertex is a usual choice– Why? You most often don’t need very many.– Also, storage space is an issue.– In practice, we’ll store N (bone index j, weight wij)

pairs per vertex.52



How to compute vertex positions?



Linear Blend Skinning• Basic Idea 1: Transform each vertex pi with each

bone as if it was tied to it rigidly.

54



Linear Blend Skinning• Basic Idea 1: Transform each vertex pi with each

bone as if it was tied to it rigidly.• Basic Idea 2: Then blend the results using the

weights.

55


“ ”MIT EECS 6.837 - Durand

Computing Vertex Positions• Basic Idea 1: Transform each vertex pi with each

bone as if it was tied to it rigidly.• Basic Idea 2: Then blend the results using the

weights.

56

p�ij = T j pi

p’ij is the vertex i transformed using bone j.Tj is the current transformation of bone j.p’i is the new skinned position of vertex i.

p�i =

�

j

wijp�ij



• Vertex p0 has weights w01=0.5, w02=0.5

57

Rest (“bind”) pose

p0

Bone 1: T1 Bone 2: T2

Computing Vertex Positions

“Skin”




• Transform by T’1 and T’2 yields p’01, p’02

58


After rotationsp0

p’01 p’02


Bone 1: T’1 Bone 2: T’2


“Skin”





• the new position is p’0=0.5*p’1 + 0.5*p’2

59


After rotationsp0

p’0


Bone 1: T’1 Bone 2: T’2


p’01 p’02

“Skin”





• the new position is p’0=0.5*p’1 + 0.5*p’2

60


After rotationsp0

p’0


Bone 1: T’1 Bone 2: T2


p’01 p’02

“Skin”

“Skin”



SSD is Not Perfect

61

After rotationsp0

q0



Questions?

62Remedy Entertainment, Microsoft Game Studios, IGN.com



Bind Pose

63

• We are given a skeleton and askin mesh in a default pose– Called “bind pose”– Undeformed vertices pi are given

in the object space of the skin• a “global” coordinate system, no hierarchy

okino.com



Bind Pose

64

• We are given a skeleton and askin mesh in a default pose– Called “bind pose”– Undeformed vertices pi are given

in the object space of the skin• Previously we conveniently

forgot that in order forp’ij = Tj pi to make sense,coordinate systems mustmatch up.

okino.com



Coordinate systems• Undeformed vertices pi are given

in the object space of the skin• Tj is in local bone coordinate system

– according to skeleton hierarchy

65



Bind Pose cont’d

66

• In the rigging phase, weline the skeleton up with theundeformed skin.– This gives some “rest pose”

bone transformations Bjfrom local bone coordinates to global

– Bj concatenates all hierarchy matrices from node j up to the root

B1

B2

Bn

....

okino.com



Bind Pose cont’d

67

• When we animate the model,the bone transformationsTj change.

okino.com



Bind Pose cont’d

68

• When we animate the model,the bone transformationsTj change.– What is Tj? It maps from the

local coordinate system ofbone j to world space.

– again, concatenates hierarchy matrices

okino.com



Bind Pose cont’d

69

• When we animate the model,the bone transformationsTj change.– What is Tj? It maps from the

local coordinate system ofbone j to world space.

• To be able to deform pi accordingto Tj, we must first express pi in the localcoordinate system of bone j.– This is where the bind pose

bone transformations Bj come in.okino.com


• To be able to deform piaccording to Tj, we mustfirst express pi in thelocal coordinate system of bone j.– This is where the bind pose

bone transformations Bj come in.


Bind Pose cont’d

70

p�ij = T jB−1j pi

This maps pi from bind pose to the local coordinate system of bone j using B-1j,

and then to world space using Tj.okino.com



Bind Pose cont’d

71



and then to world space using Tj.

What is Tj B-1j? It is the relative change between the bone

transformations between the current and the bind pose.



Bind Pose cont’d

72






What is the transformation

when the model is still in

bind pose?



Bind Pose cont’d

73






What is the transformation

when the model is still in

bind pose?The

identity!



Questions?

74



Bind pose & weights

75

• We then figure out the vertexweights wij.– How? Usually paint by hand!– We’ll look at much cooler

methods in a while.

B1

B2

Bn

....

okino.com



Questions?

76



• Do the usual forward kinematics – maybe quaternion interpolation for rotations– get a matrix Tj(t) per bone

(full transformation from local to world)• For each skin vertex pi

77

Skinning Pseudocode

p�i =�

j

wijT j(t)B−1j pi





78

Skinning Pseudocode

p�i =�

j

wijT j(t)B−1j pi

Do you remember how to treat normals?





• Inverse transpose for normals!

79

Skinning Pseudocode

p�i =�

j

wijT j(t)B−1j pi

n�i =

��

j

wijT j(t)B−1j

�−Tni



• Do the usual forward kinematics • For each skin vertex pi

• Note that the weights & bind pose vertices are constant over time– Only matrices change

(small number of them, one per bone)– This enables implementation on GPU “vertex shaders”

(little information to update for each frame)80

Skinning Pseudocode

p�i =�

j

wijT j(t)B−1j pi



Questions?

81



Hmmh...• This is what we do to get deformed positions

82

p�i =�

j

wijT j(t)B−1j pi




• But wait...

83

p�i =�

j

wijT j(t)B−1j pi

p�i =

�

j

wijT j(t)B−1j

pi




• But wait...

• This is exactly what I warned you of on Thursday: blending matrices entry-by-entry (!!!) 84

p�i =�

j

wijT j(t)B−1j pi

p�i =

�

j

wijT j(t)B−1j

pi



• From: Pose Space Deformation: A Unified Approach to Shape Interpolation and• Skeleton-Driven Deformation• J. P. Lewis, Matt Cordner, Nickson Fong

85

Indeed... Limitations• Rotations really need to be combined differently

(quaternions!)



Usual Solution• Have the artists deal with it O:-)

• In practice, build a rig that has extra bones near joints that move in sync to counter the artifacts.– Tedious, but this is what people often do.– Need sophisticated animation controls to drive this.

• Cool paper that does this automatically:Kavan, Collins, O’Sullivan: Automatic Linearization of Nonlinear Skinning, I3D 2009

86


http://doi.acm.org/10.1145/1507149.1507157

http://doi.acm.org/10.1145/1507149.1507157

http://doi.acm.org/10.1145/1507149.1507157

http://doi.acm.org/10.1145/1507149.1507157

Fredo Durand 6.837 MIT EECS

Back to quaternions!Dual quaternions!ToG 2008



Dual numberskind of like complex numbers, vector space

a dual number â=a0+εaεwhere a0 and aε are scalar

and ε2=0 then you can deduce multiplication rules by linearity



Dual quaternionsq=q0+εqε8 dimensional!Can represent rigid transformations naturally

rotation + translation

Does the right interpolation when the axis of rotation translates

^



People Have Thought of This...

90http://isg.cs.tcd.ie/projects/DualQuaternions/Tuesday, October 5, 2010

http://isg.cs.tcd.ie/projects/DualQuaternions/

http://isg.cs.tcd.ie/projects/DualQuaternions/

Real-time enveloping with rotational regressionWang, Pulli, PopovicWe learn a fast model from exported examples.

Exported Examples

(skeleton-mesh pairs)

Fast ModelOur method

Black Box Simulation

Slide from Rob Wang



Figuring out the Weights• Usual approach: Paint them on the skin.• Can also find them by optimization from example

poses and deformed skins.– Wang & Phillips, SCA 2002

92


http://doi.acm.org/10.1145/545261.545283

http://doi.acm.org/10.1145/545261.545283


Super Cool: Automatic Rigging• When you just have some reference skeleton

animation (perhaps from motion capture) and a skin mesh, figure out the bone transformations and vertex weights!

• Ilya Baran, Jovan Popovic: Automatic Rigging and Animation of 3D Characters,SIGGRAPH 2007– http://www.mit.edu/

~ibaran/autorig/

93


http://www.mit.edu/~ibaran/autorig/






First fully automatic rigging algorithm

Pinocchio

Input Processing Output

3D Character

Generic Skeleton

Rigged Character

Slide from Ilya Baran


• When you have no skeleton, but a source animation for the full mesh (not so common)


The Other Direction


transformations in ray tracing - ustcstaff.ustc.edu.cn/~lgliu/courses/computergraphics... · mit...

Documents