risikondor - university of chicagothe fourier series (1807)...
TRANSCRIPT
Intro to harmonic analysis on groupsRisi Kondor
.
The Fourier series (1807)Any (sufficiently smooth) function f on the unit circle (equivalently, any2π–periodic f ) can be decomposed into a sum of sinusoidal waves
f(x) =∞∑
k=−∞cn e
ikx cn =1
2π
∫ 2π
0f(x) e−ikx dx.
• Workhorse of much of applied mathematics.• Exact conditions under which it works get messy.
E.g., for f ∈L2([0, 2π)) almost everywhereconvergence proved only in 1966 (Carleson).
2/37...
2/37
.
The Fourier transform
f(x) =
∫f(k) e2πikx dk f(k) =
∫f(x) e−2πikx dx
• Duality between time domain and Fourier domain (wave/particle duality inquantum mechanics)
• Heisenberg uncertainty principle• Easily generalizes to Rp.
3/37...
3/37
.
The discrete Fourier transform (DFT)
f(x) =n−1∑k=0
f(k) e2πikx/n f(k) =1
n
n−1∑x=0
f(x) e−2πikx/n
• Unitary transform Cn → Cn (with appropriate normalization).• Can be seen as discretized version of Fourier series, or as the Fourier
transform on 0, 1, 2, . . . , n−1.• Foundation of all of digital signal processing.• Fast Fourier transforms reduce computation time from O(n2) toO(n log n) [Cooley & Tukey, 1965].
4/37...
4/37
Underlying principles
.
1. Analytic
Take a measurable spaceX , a space of functions onX , say L2(X), and aself-adjoint smoothing operator Υ. For example, onX =Rp,Υ may be thetime t diffusion operator
(Υf)(x) =1√4πt
∫f(y) e−∥x−y ∥2/(4t)dy.
Question: How doesΥ filter L2(X) into a nested sequence of spaces
WΩ = f ∈L2(X) | |⟨f,Υf⟩ / ⟨f, f⟩| ≤ Ω ?
6/37...
6/37
.
2. Algebraic
Now let a groupG act onX inducing linear operators Tg : L2(X) → L2(X).E.g., on onX =Rp,
(Tgf)(x) = f(x− g) g ∈Rp.
Question: What are the smallest spaces fixed by these operators,
Tg(V ) = V ∀g ∈G ?
7/37...
7/37
.
On Rp we are lucky because these two notions match up:
• The diffusion operator is et∇2, where∇2 is the Laplacian
∇2 =∂2
∂x21+
∂2
∂x22+ . . .+
∂2
∂x2p.
• The e2πik·x Fourier basis functions are eigenfunctions of both∆ and Tg : ∇2e2πik·x = −4π2∥k∥2 e2πik·x, Tg e
2πik·x = e2πik·g e2πik·x.
• Therefore WΩ = f | f(k) = 0 if ∥k∥2 ≥Ω (band-limited functions)
Vκ = f | f(k) = 0 if k = κ (isotypics)
Question: Does this correspondence hold more generally?
8/37...
8/37
.
Fourier analysis on graphs
On a finite graph G, the analog of∆ is the graph Laplacian
[L]i,j =
1 i ∼ j
− di i= j
0 otherwise.
It does lead to a natural measure of smoothness:
f⊤Lf = −∑i∼j
(fi − fj)2.
Analyzing functions in terms of the eigenfunctions of L is called spectral graphtheory.
However (in general) on graphs there is no analog of translation.
9/37...
9/37
More properties of the FT on R
.
• The Fourier transform is Linear Invertible
∫f(x)g(x)∗dx =
∫f(k) g(k)∗dk (Parseval thm)
Therefore, it is essentially a unitary change of basis.
11/37...
11/37
.
• Diagonalizes the derivative operator:
g(x) =d
dxf(x) =⇒ g(k) = 2πik f(k).
• Diagonalizes the Laplacian:
g(x) =d2
dx2f(x) =⇒ g(k) = −4πk2 f(k).
12/37...
12/37
.
• Translation theorem:
g(x) = f(x− t) =⇒ g(k) = e−2πikt f(k)
• Scaling theorem:
g(x) = f(λx) =⇒ f ′(k) = |λ|−1 f(k/λ)
13/37...
13/37
.
• Convolution theorem:
(f ∗ g)(x) =∫f(x−y)g(y)dy =⇒ f ∗g(k) = f(k) · g(k)
• Cross-correlation theorem:
(f ⋆ g)(x) =
∫f(y)∗g(x+y)dy =⇒ f ⋆ g(k) = f(k)∗ · g(k)
Autocorrelation:
h(x) =
∫f(y)∗f(x+y)dy =⇒ h(k) = ∥f(k)∥2
14/37...
14/37
Fourier analysis on compact groups
.
Fourier tranform on R
f(k) =
∫f(x) e−2πikx dx
Observation: χk(x) = e−2πikx are exactly the characters of R.
16/37...
16/37
.
Locally Compact Abelian groups
The Fourier transform of a function on an LCA groupG with Haar measure µ is
f(χ) =
∫Gf(x)χ(x) dµ χ∈ G.
• The dual object is itself a group: T ↔ Z, R ↔ R, and for finite groupsG ∼= G (Pontryagin duality).
• This covers the Fourier series and the Fourier transform.
17/37...
17/37
.
Compact non-Abelian groups
The Fourier transform of a function on a compact groupG with Haar measure µis
f(ρ) =
∫Gf(x) ρ(x) dµ(x) ρ ∈ R,
whereR is a complete set of inequivalent irreducible representations (irreps).
• Now the dual object is no longer a group, but a set of representations(Tannaka–Krein duality).
• IfG is finite,R is finite. IfG is compact,R is countable.• Each Fourier component f(ρ) is a matrix.
In the following, we will always assume that each ρ is unitary. Everyrepresentation is over C.
18/37...
18/37
Properties
.
InvertibilityForward transform:
f(ρ) =
∫Gf(x) ρ(x) dµ(x) ρ ∈ R.
Inverse transform:
f(x) =1
µ(G)
∑ρ∈R
dρ tr[f(ρ) ρ(x−1)
]x∈G.
• Just as before (with respect to the appropriate scaled matrix norms), thistransform is unitary.
• The eρi,j(x) =√dρ [ρ(x)]i,j functions form an orthogonormal basis
(Peter-Weyl theorem).
20/37...
20/37
.
Left-translation
Theorem. Given f : G→ C and t∈G, define f t(x) = f(t−1x). Then
f t(ρ) = ρ(t) · f(ρ) ρ∈R.
Proof.∫f t(x) ρ(x) dµ(x) =
∫f(t−1x) ρ(x) dµ(x) =
∫f(x) ρ(tx) dµ(x) =
∫f(x) ρ(t) ρ(x) dµ(x) = ρ(t) f(ρ)
21/37...
21/37
.
Left-translation
• Convolution theorem:
(f ∗ g)(x) =∫f(xy−1)g(y)dµ(y) =⇒ f ∗g(ρ) = f(ρ) · g(ρ)
• Cross-correlation theorem:
(f ⋆ g)(x) =
∫f(xy)g(y)∗dµ(y) =⇒ f ⋆ g(ρ) = f(ρ) · g(ρ)†
22/37...
22/37
.
Right-translation
Theorem. Given f : G→ C and t∈G, define f (t)(x) = f(xt−1). Then
f (t)(ρ) = f(ρ) · ρ(t) ρ∈R.
Proof.∫f (t)(x) ρ(x) dµ(x) =
∫f(xt−1) ρ(x) dµ(x) =
∫f(x) ρ(xt) dµ(x) =
∫f(x) ρ(x) ρ(t) dµ(x) = f(ρ) ρ(t)
23/37...
23/37
.
Invariant subspaces
• To left-translation:
Wρ,j = span eρi,j | j = 1, . . . , dρ ρ∈R j = 1, . . . , dρ.
• To right-translation:
Wρ,i = span eρi,j | i = 1, . . . , dρ ρ∈R i = 1, . . . , dρ.
• To left- and right-translation:
Vρ = span eρi,j | i, j = 1, . . . , dρ ρ∈R.
24/37...
24/37
.
The group algebra
The group algebra C[G] is a space with orthonormal basis ex | x∈G anda notion of multipilication defined by
exey = exy ∀x, y ∈G.
Letting ⟨f, ex⟩ = f(x) and extending to the rest of C[G] by linearity, for anyf, g ∈C[G],
(fg)(x) =
∫f(xy−1)g(y)dµ(y) = (f ∗ g)(x).
The group algebra of any compact group is semi-simple, i.e., it decomposesinto a direct sum of simple algebras.
25/37...
25/37
.
The group algebra
• The group algebra decomposes into a sum of simple algebras:
C[G] =⊕ρ
Vρ. (1)
Each Vρ is called an isotypic, and corresponds to a single Fourier matrixf(ρ). This decomposition is unique.
• Each Vρ further decomposes into a sum of dρ leftG–modules
Vρ =W ρ1 ⊕W ρ
2 ⊕ . . .⊕W ρdρ
(2)
corresponding to each column of f(ρ). This decomposition is not unique,i.e., it depends on the choice ofR.
The Fourier transform is a projection of f onto a basis adapted to (1) and (2).
26/37...
26/37
Fourier analysis on homogeneous spaces
.
Group actions
• So far we have considered: f is a function on a compact groupG. G acts onG by t : x 7→ tx inducing f 7→ f t, where f t(x) = f(t−1x)
(similarly for the right-action and right-translation).
• In practice it is often more common that: f is a function on a setX . G acts onX transitively by t : x 7→ tx, inducing f 7→ f t, wheref t(x) = f(t−1x).
Example: The rotation group SO(3) and the sphere S2. The symmetric groupacting on a matrix by permuting rows/columns.
28/37...
28/37
.
Homogeneous spacesAssume thatG acts onX transitively by x 7→ gx.
• Pick some x0 ∈X .• The subset ofG fixing x0 is a subgroupH ofG.• Each set gH = gh | h∈H is called a leftH–coset.• The set of leftH–cosets we denoteG/H .• gH | gH ∈G/H forms a partition ofG.• yx0 = y′x0 if and only if y, y′ belong to the same coset.• Therefore, we have a bijection
X ↔ G/H.
X is called a homogeneous space ofG.
Example: S2 ∼ SO(3)/SO(2).
29/37...
29/37
.
FT on homogeneous spaces
Now L(X) is only aG-module, not a C[G] algebra. However, we can still ask,how it decomposes into a sum of invariant modules.
Definition. The Fourier transform of f : X→C is the FT of the inducedfunction f ↑G(g) = f(gx0), i.e.,
f(ρ) =
∫Gf(gx0) ρ(g) dµ(g) ρ ∈ R
30/37...
30/37
.
Adapted bases
We say that the repesentation ρ ofG is adapted toH ≤ G, if ρ↓H is of theblock diagonal form
ρ↓H(h) =⊕ρ′∈Rρ
ρ′(h) h∈H (3)
for some multiset Rρ of irreps ofH .
We usemtr(ρ) to denote the multiplicity of the trivial representation in thedecomposition (3).
31/37...
31/37
.
FT on homogeneous spaces
Theorem. If f : X → C and f is expressed in a basis adapted toH ≤ G,then f(ρ) has at mostmtr(ρ) non-zero columns.
Proof.
f(ρ) =
∫g∈G/H
∫h∈H
f(gx0) ρ(gh) dµ(g)dµ(h) =
[∫g∈G/H
f(gx0) ρ(g) dµ(g)
][∫h∈H
ρ(h)dµ(h)︸ ︷︷ ︸]
32/37...
32/37
.
Proof (continued)∫h∈H
ρ(h)dµ(h) =⊕ρ′∈R
∫h∈H
ρ′(h)dµ(h)
If ρ′ is the trivial irrep ofH , then∫h∈H ρ
′(h)dµ(h) = µ(H). However, byorthogonality of the Fourier basis functions, for any other irrep,∫h∈H ρ
′(h)dµ(h) = 0.
33/37...
33/37
.
ExampleThe rotation group SO(3) is parametrized by the Euler angles (θ, ϕ, ψ), andthe irreps are given by theD(0), D(1), D(2), . . .Wigner matrices, where
[D(ℓ)]m,m′ = e−im′ψ Y m
ℓ (θ, ϕ), m,m′ = −ℓ, . . . , ℓ
where
Y mℓ (θ, ϕ) =
√2ℓ+ 1
4π
(ℓ−m)!
(ℓ+m)!Pml (cos θ) eimϕ,
are the spherical harmonics. Clearly, the Wigner matrices are adapted to thesubgroup SO(2) that rotates ψ. In particular,
D(ℓ) ↓SO(2) (ψ) =
ℓ⊕m′=−ℓ
χm′(ψ) χm′(ψ) = e−im′ψ,
so the multiplicity of the trivial irrep χ0 is 1.
34/37...
34/37
.
Example
The Fourier transform of f : S2 → C is
f(ℓ) =
∫SO(3)
f(Rx0)D(ℓ)(R) dµ(R) ℓ = 0, 1, 2, . . . ,
but by our theorem only the middle column of each of these matrices isnon-zero, which yields exactly the celebrated spherical harmonic expansion
f(θ, ϕ) =∞∑ℓ=0
m∑m=−ℓ
fmℓ Y mℓ (θ, ϕ).
35/37...
35/37
.
Example
36/37...
36/37
.
APPLICATIONS
• Invariants to group actions: Computer vision Graph invariants
• Band–limited approximations on Sn: Multi-object tracking
• Wavelets on Sn• Learning on Sn:
Ranking problems
• Optimization on Sn: Fast QAP solvers.
37/37...
37/37