Wavelets & Wavelet Algorithms

Vladimir Kulyukin

www.vkedco.blogspot.comwww.vkedco.blogspot.com

1D Fourier Analysis

Outline

● Review● 1D Sinusoid Data Generation with Anonymous

Functions in Octave/Matlab● 1D Fourier Coefficients & Phases● 1D Fourier Analysis Algorithm

Review

Common Harmonic Function Form

.sincosThen .cos and sinLet

.sincoscossin

sincoscossinsin

:have weformula, Sum Angle of Sine theUsing

.sinLet

tbtatxAbAa

tAtA

ttAtA

tAtx

Common Harmonic Function Form

.tansin Thus,

. and tan

:follows as and get can we, and given are weSince

.harmonic is sincosfunction Every

122

221

b

atbatx

baAb

a

Aba

tbtatx

.sincos

sincoscossincossinsincos

sintansin :onVerificati 122

tbta

tAtAttA

tAb

atbatx

Periodic Harmonics

;4

sin4

cos

;3

sin3

cos

;2

sin2

cos

;sincos

:Examples

,...3,2,1 ,sincos

form theof harmonics heConsider t .2Let

444

333

222

111

l

tb

l

tatx

l

tb

l

tatx

l

tb

l

tatx

l

tb

l

tatx

kl

ktb

l

ktatx

lP

kkk

Periodic Harmonics

period. a also is period a of multiple

integralany because, of period a is 2,,2 Since

.2222

why.is Here .2 period a has

,...3,2,1 ,sincos form theof harmonicAny

txlPZkkTl

lkTk

l

lk

Tl

k

lP

kl

ktb

l

ktatx

kk

kk

kk

kkk

Trigonometric Polynomials of Order n

.order of polynomial tric trigonomea is

period. a isnumber

any for which function a isconstant a because , of period a is 2

constant. a is where,sincos

:sum following heConsider t

1

nts

tsl

Al

ktb

l

ktaAts

n

n

n

kkkn

Infinite Trigonometric Series

constant. a is where,sincos1

Al

ktb

l

ktaAts

kkk

Definition of Function Orthogonality

b

a

dxxgxfxgxf 0 if orthogonal are , Functions

Orthogonality of SIN(X) & COS(X)

0cossinbecause ,orthogonal are cos,sin2

0

dxxxxx

lar.perpendicuremain always velocity rotational same the

with circle thearound rotating ,cos and sin arrows, Two xx

Orthogonality of SIN(X) & COS(X)

xsin

xcos

Integration by Summation

● Integrating functions by hand is fun but a) error-prone and b) difficult (unless you are a math major :-))

● However, integration of many sinusoids and many other useful functions can be approximated with summations, i.e., for-loops

● Computing summations makes doing integrals by hand less important than it used to be

Integration by Summation

0cossin2

0

dxxx

t = 0:0.001:2*pi;figure;plot(t, sin(t).*cos(t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-2 2]);title('sin(x)cos(x)');sum01 = sum(sin(t).*cos(t));display(strcat('SUM01 = sin(x)cos(x) = ', num2str(sum01)));

Output: SUM01 = sin(x)cos(x) on [0, 2pi] =-7.5484e-05

Orthogonality of Basic Trigonometric System

.2,over 0 is

system tric trigonomebasic theof functionsdifferent two

any of integral that theshow formulasn integratio The

,...sin,cos,...,2sin,2cos,sin,cos,1

functions ofset infinite theis system tric trigonomebasic The

aa

nxnxxxxx

Integration of Trigonometric Series

.22

0012

sincos12

:,over integrate usLet

.sincos2

:by term termintegrableexpansion series rictrigonomet

has and 2 period offunction integrablean is Let

000

1

0

1

0

1

0

aa

xa

badxa

dxkxbdxkxadxa

dxxf

xf

kxbkxaa

xf

xf

kkk

kkk

kkk

Computing Cosine Coefficients

.coscoscos

cossincoscoscos2

cos

:,over integrate and cosby sidesboth multiply usLet

.sincos2

:by term termintegrable

expansion series tric trigonomea has and 2 period offunction integrablean is Let

2

1

0

1

0

nnn

kkk

kkk

adxnxadxnxnxa

dxnxkxbdxnxkxadxnxa

dxnxxf

nx

kxbkxaa

xf

xf

,...3,2,1,cos

1ndxnxxfan

Computing Sine Coefficients

.sinsinsin

sinsinsincossin2

sin

:,over integrate and sinby sidesboth multiply usLet

.sincos2

:by term termintegrableexpansion

series tric trigonomea has and 2 period offunction integrablean is Let

2

1

0

1

0

nnn

kkk

kkk

bdxnxbdxnxnxb

dxnxkxbdxnxkxadxnxa

dxnxxf

nx

kxbkxaa

xf

xf

,...3,2,1,sin

1ndxnxxfbn

Fourier Coefficients

,...3,2,1,sin

1ndxnxxfbn

,...3,2,1,cos

1ndxnxxfan

Fourier Series

. of seriesFourier thecalled

is sincos2

series tric trigonomeThe

,...3,2,1,sin1

and cos1

where,sincos2

:expansion series

tric trigonomefollowing thehas and 2 period offunction a is If

1

0

1

0

xf

kxbkxaa

nnxxfbnxxfa

kxbkxaa

xf

xf

kkk

nn

kkk

1D Sinusoid Generationwith

Anonymous Functions

Warmup Problem

.22

1,

2

1,2,0,,:intervals three theof one is ,

and 3,2,1 where,sin where,

esapproximat that program labOctave/Matan Write

ba

ttfdxtfb

a

Interval on [-PI, PI]

DELTA = 0.001;t = -pi:DELTA:pi;w1 = 1;w2 = 2;w3 = 3;%% this is how you define an anonymous function %% and apply it to the t array to get datadata01_on_t = arrayfun(@(x) sin(w1*x), t);data02_on_t = arrayfun(@(x) sin(w2*x), t);data03_on_t = arrayfun(@(x) sin(w3*x), t);

SIN(t) on [-PI, PI]

figure;plot(t, data01_on_t);xlabel('x');ylabel('y');title('f(t) = sin(t) on [-pi, pi]');sum_data01_on_t = sum(data01_on_t);display(strcat('sum_data01_on_t = ', num2str(sum_data01_on_t)));

SIN(2t) on [-PI, PI]

figure;plot(t, data02_on_t);xlabel('x');ylabel('y');title('f(t) = sin(2t) on [-pi, pi]');sum_data02_on_t = sum(data02_on_t);display(strcat('sum_data02_on_t = ', num2str(sum_data02_on_t)));

SIN(3t) on [-PI, PI]

figure;plot(t, data03_on_t);xlabel('x');ylabel('y');title('f(t) = sin(3t) on [-pi, pi]');sum_data03_on_t = sum(data03_on_t);display(strcat('sum_data03_on_t = ', num2str(sum_data03_on_t)));

Interval [0, 2PI]

DELTA = 0.001;t2 = 0:DELTA:2*pi;w1 = 1;w2 = 2;w3 = 3;%% this is how you define an anonymous function %% and apply it to the t array to get datadata01_on_t2 = arrayfun(@(x) sin(w1*x), t2);data02_on_t2 = arrayfun(@(x) sin(w2*x), t2);data03_on_t2 = arrayfun(@(x) sin(w3*x), t2);

SIN(t) on [0, 2PI]

figure;plot(t2, data01_on_t2);xlabel('x');ylabel('y');title('f(t) = sin(t) on [0, 2pi]');sum_data01_on_t2 = sum(data01_on_t2);display(strcat('sum_data01_on_t2 = ', num2str(sum_data01_on_t2)));

SIN(2t) on [0, 2PI]

figure;plot(t2, data02_on_t2);xlabel('x');ylabel('y');title('f(t) = sin(2t) on [0, 2pi]');sum_data02_on_t2 = sum(data02_on_t2);display(strcat('sum_data02_on_t2 = ', num2str(sum_data02_on_t2)));

SIN(3t) on [0, 2PI]

figure;plot(t2, data03_on_t2);xlabel('x');ylabel('y');title('f(t) = sin(3t) on [0, 2pi]');sum_data03_on_t2 = sum(data03_on_t2);display(strcat('sum_data03_on_t2 = ', num2str(sum_data03_on_t2)));

Interval [0.5, 0.5+2PI]

DELTA = 0.001;t3 = 0.5:DELTA:(0.5+2*pi);w1 = 1;w2 = 2;w3 = 3;%% this is how you define an anonymous function %% and apply it to the t array to get datadata01_on_t3 = arrayfun(@(x) sin(w1*x), t3);data02_on_t3 = arrayfun(@(x) sin(w2*x), t3);data03_on_t3 = arrayfun(@(x) sin(w3*x), t3);

SIN(t) on [0.5, 0.5+2PI]

figure;plot(t3, data01_on_t3);xlabel('x');ylabel('y');title('f(t) = sin(t) on [0.5, 0.5+2pi]');sum_data01_on_t3 = sum(data01_on_t3);sum_data01_on_t3 = sum_data01_on_t3*DELTA;display(strcat('sum_data01_on_t3 = ', num2str(sum_data01_on_t3)));

SIN(2t) on [0.5, 0.5+2PI]

figure;plot(t3, data02_on_t3);xlabel('x');ylabel('y');title('f(t) = sin(2t) on [0.5, 0.5+2pi]');sum_data02_on_t3 = sum(data02_on_t3);sum_data02_on_t3 = sum_data02_on_t3*DELTA;display(strcat('sum_data02_on_t3 = ', num2str(sum_data02_on_t3)));

SIN(3t) on [0.5, 0.5+2PI]

figure;plot(t3, data03_on_t3);xlabel('x');ylabel('y');title('f(t) = sin(3t) on [0.5, 0.5+2pi]');sum_data03_on_t3 = sum(data03_on_t3);sum_data03_on_t3 = sum_data03_on_t3*DELTA;display(strcat('sum_data03_on_t3 = ', num2str(sum_data03_on_t3)));

1D Fourier Coefficients & Phases

Motivation

● Let us assume that there exists some signal function f(t) that generates some data (for now, it is 1D data)

● This signal function may or may not be known to the researcher; typically it is unknown and accessible only through the generated data

● We would like to extract the Fourier coefficients and phases from the 1D data computed by f(t): this is what is known as conversion from time domain into frequency domain

Problem 01

.2 islength whoseintervalsdifferent on three

formulaeFourier the validate tois objectiveOur .function signal

theknow that we,simplicity of sake for the assume, wenow,For

.,on of tscoefficienFourier thecompute toprogram

labOctave/Matan Write.3sin4

12sin

2

1sinLet

tf

tf

ttttf

Problem 01: Sinusoid PlotDELTA = 0.001;t = -pi:DELTA:pi; %% t axis [-pi, pi] in increments of DELTAb1 = 1;b2 = 0.5;b3 = 0.25;w1 = 1;w2 = 2;w3 = 3;data01 = arrayfun(@(x) b1*sin(w1*x)+b2*sin(w2*x)+b3*sin(w3*x), t);figure;plot(t, y);xlabel('x');ylabel('y');title('f(t) = sin(x) + 0.5sin(2x) + 0.25sin(3x)');

Problem 01

,on 3sin4

12sin

2

1sin ofPlot ttttf

Problem 01: Computing Cosine Coefficients

%% prefix 'aa' means 'approximation of a'aa0 = 1/pi*sum(data01.*cos(0*t));aa0 = aa0*DELTA;aa1 = 1/pi*sum(data01.*cos(w1*t));aa1 = aa1*DELTA;aa2 = 1/pi*sum(data01.*cos(w2*t));aa2 = aa2*DELTA;aa3 = 1/pi*sum(data01.*cos(w3*t));aa3 = aa3*DELTA;

,...3,2,1,cos

1ndxnxxfan

Problem 01: Computing Sine Coefficients

ab1 = 1/pi*sum(data01.*sin(w1*t)); %% get the b coeff that corresponds to w1 ab1 = ab1*DELTA; %% if we do not multiply by DELTA ab1 == 1000ab2 = 1/pi*sum(data01.*sin(w2*t));ab2 = ab2*DELTA;ab3 = 1/pi*sum(data01.*sin(w3*t));ab3 = ab3*DELTA; %% These coefficients will not be present, because their corresponding%% frequencies are not in the signal sinusoidcoeff_map = containers.Map('KeyType', 'int32', 'ValueType', 'any');for freq = 4:1:10 coeff_map(freq) = 1/pi*sum(y.*sin(freq*x));end

Problem 01: Theoretical & Approximated Coefficients

>>aa0 =1.8021e-08

>> aa1 =-1.8021e-08

>> aa2 =1.8021e-08

>> aa3 = -1.8021e-08

>> ab1 =1.0000

>> ab2 =0.5000

>> ab3 =0.2500

25.0

5.0

1

;3sin4

12sin

2

1sin

3

2

1

b

b

b

ttttf

Theoretical Coeffs Approximated Coeffs

Problem 02

0.001. ofincrement an with

edapproximat , interval on the 50] [1,in frequencesany has ifout

find tolike would We.function signal unknown"" someby generated

ata,sinusoid_d called array, data 1D thehaveonly that weassume usLet

tf

tf

Problem 02: Computing & Plotting Sine Coeffs

sine_coeff_map = containers.Map('KeyType', 'int32', 'ValueType', 'any');for f1 = 1:1:50 sine_coeff_map(f1) = 1/pi*sum(sinusoid_data.*sin(f1*t))*DELTA;endfigure;plot(1:1:50, arrayfun(@(f) sine_coeff_map(f), 1:1:50));xlabel('Freq');ylabel('Sine Coeff');

,...3,2,1,sin1

,cos1

where

,sincos2

: of seriesFourier 1

0

nnxxfbnxxfa

kxbkxaa

xfxf

nn

kkk

Problem 02: Computing & Plotting Sine Coeffs

This plot is generated by the code on the previous slide

Problem 02: Computing & Plotting Cosine Coeffs

cosine_coeff_map = containers.Map('KeyType', 'int32', 'ValueType', 'any');for f2 = 0:1:50 cosine_coeff_map(f2) = 1/pi*sum(sinusoid_data.*cos(f2*t))*DELTA;endfigure;plot(0:1:50, arrayfun(@(f) cosine_coeff_map(f), 0:1:50));xlabel('Freq');ylabel('Cosine Coeff');

,...3,2,1,sin1

,cos1

where

,sincos2

: of seriesFourier 1

0

nnxxfbnxxfa

kxbkxaa

xfxf

nn

kkk

Problem 02: Computing & Plotting Cosine Coeffs

This plot is generated by the code on the previous slide

Problem 02: Looking Up Cosine Coeffs

>> cosine_coeff_map(0)=1.5886

>> cosine_coeff_map(10)=4.5178

>> cosine_coeff_map(20)=12.3378

>> cosine_coeff_map(30) = 38.0054

>> cosine_coeff_map(40)=5.9178

>> cosine_coeff_map(50)=7.3578

Problem 02: Looking Up Sine Coeffs

>> sine_coeff_map(10)=0.5000

>> sine_coeff_map(20)=1.7300

>> sine_coeff_map(30)=2.4350

>> sine_coeff_map(40)=10.7799

>> sine_coeff_map(50)=3.4779

Problem 02: Looking Up Sine Coeffs

>> sine_coeff_map(10)=0.5000

>> sine_coeff_map(20)=1.7300

>> sine_coeff_map(30)=2.4350

>> sine_coeff_map(40)=10.7799

>> sine_coeff_map(50)=3.4779

Problem 02: The “Unknown” Signal Function

>> sine_coeff_map(10)=0.5000>> sine_coeff_map(20)=1.7300>> sine_coeff_map(30)=2.4350>> sine_coeff_map(40)=10.7799>> sine_coeff_map(50)=3.4779

>> cosine_coeff_map(0)=1.5886>> cosine_coeff_map(10)=4.5178>> cosine_coeff_map(20)=12.3378>> cosine_coeff_map(30) = 38.0054>> cosine_coeff_map(40)=5.9178>> cosine_coeff_map(50)=7.3578

%%% sine coefficientsb10 = 0.5;b20 = 1.73;b30 = 2.435;b40 = 10.78;b50 = 3.478; %%% cosine coefficientsa10 = 4.5;a20 = 12.32;a30 = 37.9876;a40 = 5.90;a50 = 7.34; %% combined sinusoidA = pi/4;sine_curve = arrayfun(@(x) b10*sin(w10*x)+b20*sin(w20*x)+b30*sin(w30*x)+b40*sin(w40*x)+b50*sin(w50*x), t);cosine_curve = arrayfun(@(x) A+a10*cos(w10*x)+a20*cos(w20*x)+a30*cos(w30*x)+a40*cos(w40*x +a50*cos(w50*x), t);sinusoid_data = sine_curve + cosine_curve;

1D Fourier Analysis Algorithm

1D Fourier Analysis

● Given a 1D data array, determine the frequency range [W

lower, W

upper]

● Compute the cosine and sine coefficients for each frequency value in the frequency range

● Once the sine and cosine coefficients are computed, determine the amplitude and phase for every constituent harmonic

● Optional: If the signal function is known, recombine the reconstructed harmonics and compute how closely their sum approximates the signal function

1D Fourier Analysis: Pseudocode

.sin

harmonicth - therepresents ,, tuple-3 The

}

container; somein ,, Save

;tan

;sin1

;cos1

{in every For

., e.g., range,frequency a be Let

.:001.0: e.g., interval, timea be Let

function. signal some from valuesofarray 1D a be Let

22

1

kkkk

kkk

kkk

k

kk

kkkk

k

upperlower

atabaa

kaabaa

aabaa

ab

aaa

tdataabtdataaa

W

WWWW

tt

data

Problem 03

0.001. of incrementsin dconstructe ,on of analysisFourier 1D Do

.57.30,98.7,73.1,32.12,5.0,5.4,42

where

,20sin573030cos987

20sin73120cos3212

10sin5010cos544

Let

3322110

tf

bababaa

t.t.

t.t.

t.t.π

tf

Solution Steps

● Generate 1D data (this step is unnecessary if the data array is given)

● Approximate sine & cosine coefficients● Reconstruct 10th, 20th, and 30th harmonics● Combine reconstructed harmonics to reconstruct

the original sinusoid● Compute approximate error b/w reconstructed &

original sinusoids

Problem 03: Generate 1D Data

DELTA = 0.001;t = -pi:DELTA:pi;%% frequenciesw10 = 10;w20 = 20;w30 = 30;%%% sine coefficientsb10 = 0.5;b20 = 1.73;b30 = 30.57;%%% cosine coefficientsa10 = 4.5;a20 = 12.32;a30 = 7.98; %% combined sinusoidA = pi/4;sine_curve = arrayfun(@(x) b10*sin(w10*x)+b20*sin(w20*x)+b30*sin(w30*x), t);cosine_curve = arrayfun(@(x) A+a10*cos(w10*x)+a20*cos(w20*x)+a30*cos(w30*x), t);sinusoid_data = sine_curve+cosine_curve;

Problem 03: Approximate Cosine Coeffs

%% approximating cosine coeffs%% this can also be done with a for-loop %% iteration over the frequency rangeaa0 = 1/pi*sum(sinusoid_data.*cos(0*t));aa0 = aa0*DELTA;aa10 = 1/pi*sum(sinusoid_data.*cos(w10*t));aa10 = aa10*DELTA;aa20 = 1/pi*sum(sinusoid_data.*cos(w20*t));aa20 = aa20*DELTA;aa30 = 1/pi*sum(sinusoid_data.*cos(w30*t));aa30 = aa30*DELTA;

Problem 03: Approximate Sine Coeffs

%% approximating sine coeffsab10 = 1/pi*sum(sinusoid_data.*sin(w10*t)); ab10 = ab10*DELTA;ab20 = 1/pi*sum(sinusoid_data.*sin(w20*t));ab20 = ab20*DELTA;ab30 = 1/pi*sum(sinusoid_data.*sin(w30*t));ab30 = ab30*DELTA;

Problem 03: Reconstruct 10th Harmonic

%% amplitude & phaseaA10 = sqrt(aa10^2 + ab10^2);aPhi10 = atan(aa10/ab10); %% two different formulas for 10th harmonicH10 = aa10*cos(w10*t) + ab10*sin(w10*t);HH10 = aA10*sin(w10*t + aPhi10);

101010

101010

10

10110

210

21010

10sin

;10sin10cos

tan ;

tAtHH

tbtatH

b

abaA

Problem 03: Reconstruct 20th Harmonic

%% amplitude & phaseaA20 = sqrt(aa20^2 + ab20^2);aPhi20 = atan(aa20/ab20); %% two different formulas for H20H20 = aa20*cos(w20*t) + ab20*sin(w20*t);HH20 = aA20*sin(w20*t + aPhi20);

202020

202020

20

20120

220

22020

20sin

;20sin20cos

tan ;

tAtHH

tbtatH

b

abaA

Problem 03: Reconstruct 30th Harmonic

%% amplitude & phaseaA30 = sqrt(aa30^2 + ab30^2);aPhi30 = atan(aa30/ab30); %% two different formulas for H30H30 = aa30*cos(w30*t) + ab30*sin(w30*t);HH30 = aA30*sin(w30*t + aPhi30);

303030

303030

30

30130

230

23030

30sin

;30sin30cos

tan ;

tAtHH

tbtatH

b

abaA

Problem 03: Recombine & Plot Harmonics

rH = aa0/2 + H10 + H20 + H30;figure;plot(t, rH);xlabel('t');ylabel('rH');title('rH=aa0/2 + H10 + H20 + H30'); rHH = aa0/2 + HH10 + HH20 + H30;figure;plot(t, rHH);xlabel('t');ylabel('rHH');title('rHH=aa0/2 + HH10 + HH20 + HH30');

Problem 03: Compare Original & Reconstructed Functions

Original Sinusoid Reconstructed Sinusoid rH

Problem 03: Compare Original & Reconstructed Functions

Original Sinusoid Reconstructed Sinusoid rHH

Problem 03: Compute Approximation Error

percent_error_rH = sum(abs(abs(rH) - abs(sinusoid_data)))/sum(abs(sinusoid_data));percent_error_rHH = sum(abs(abs(rHH) - abs(sinusoid_data)))/sum(abs(sinusoid_data));disp(strcat('%error(sinusoid_data,rH)=', num2str(percent_error_rH)));disp(strcat('%error(sinusoid_data,rHH)=', num2str(percent_error_rHH)));

%error(sinusoid_data,rH)=0.00028033%error(sinusoid_data,rHH)=0.00028033

References● J. O. Smith III, Mathematics of the Discrete Fourier Tranform with

Audio Applications, 2nd Edition.

● G. P. Tolstov. Fourier Series.