course note signals

8/8/2019 Course Note Signals

1/122

Discrete-timeSignalsandSystems


2/122

ii


3/122

Discrete-timeSignalsandSystemsAnOperatorApproach

SanjoyMahajanandDennisFreemanMassachusettsInstituteofTechnology


4/122

TypesetinPalatinoandEulerbytheauthorsusingConTEXtandPDFTEX

Copyright2009SanjoyMahajanandDennisFreemanSourcerevision:66261db0f9ed+ (2009-10-18 13:33:48 UTC)Discrete-timeSignalsandSystemsbySanjoyMahajanandDennisFreeman(authors)and??(publisher)islicensedunderthe. . . license.

C


5/122

Briefcontents

Preface ix1 Differenceequations 12 Differenceequationsandmodularity 173 Blockdiagramsandoperators:Twonewrepresentations 334 Modes 515 Repeatedroots 636 Theperfect(sine)wave 717 Control 838 Proportionalandderivativecontrol 95

Bibliography 105Index 107


6/122

vi


7/122

Contents

Preface ix1 Differenceequations 1

1.1 Rabbits 21.2 Leakytank 71.3 Fallofafogdroplet 111.4 Springs 14

2 Differenceequationsandmodularity 172.1 Modularity:Makingtheinputliketheoutput 172.2 Endowmentgift 212.3 Rabbits 25

3 Blockdiagramsandoperators:Twonewrepresentations 333.1

Disadvantages

of

difference

equations

34

3.2 Blockdiagramstotherescue 353.3 Thepowerofabstraction 403.4 Operationsonwholesignals 413.5 Feedbackconnections 453.6 Summary 49

4 Modes 514.1 GrowthoftheFibonacciseries 524.2 TakingoutthebigpartfromFibonacci 554.3 Operatorinterpretation 574.4 Generalmethod:Partialfractions 59

5 Repeatedroots 635.1 Leaky-tankbackground 645.2 Numericalcomputation 655.3 Analyzingtheoutputsignal 67


8/122

viii

5.4 Deformingthesystem:Thecontinuityargument 685.5 Higher-ordercascades 70

6 Theperfect(sine)wave 716.1 ForwardEuler 726.2 BackwardEuler 766.3 Leapfrog 796.4 Summary 82

7 Control 837.1 Motormodelwithfeedforwardcontrol 837.2 Simplefeedbackcontrol 857.3 Sensordelays 877.4 Inertia 90

8 Proportionalandderivativecontrol 958.1 Whyderivativecontrol 958.2 Mixingthetwomethodsofcontrol 968.3 Optimizingthecombination 988.4 Handlinginertia 998.5 Summary 103Bibliography 105Index 107


9/122

Preface

Thisbookaimstointroduceyoutoapowerfultoolforanalyzinganddesigningsystemswhetherelectronic,mechanical,orthermal.ThisbookgrewoutoftheSignalsandSystemscourse(numbered6.003)thatwehavetaughtonandofftoMITsElectricalEngineeringandComputerSciencestudents.The traditional signals-and-systems course for example [17] emphasizestheanalysisofcontinuous-timesystems,inparticularanalogcircuits.However,mostengineerswillnotspecializeinanalogcircuits.Rather,digitaltechnologyofferssuchvastcomputingpowerthatanalogycircuitsareoftendesignedthroughdigitalsimulation.Digitalsimulationisaninherentlydiscrete-timeoperation. Furthermore,almostallfundamentalideasofsignalsandsystemscanbetaughtusingdiscrete-timesystems. Modularityandmultiplerepresentations , forexample,aidthedesignofdiscrete-time(orcontinuous-time)systems.Similarly,theideasformodes,poles,control,andfeedback.Furthermore,byteachingthematerialinacontextnotlimitedtocircuits,weemphasizethegeneralityofthesetools.Feedbackandsimulationaboundinthenaturalandengineeredworld,andwewouldlikeourstudentstobeflexibleandcreativeinunderstandinganddesigningthesesystems.Therefore,webeginour SignalsandSystemscoursewithdiscrete-timesystems,andgiveourstudentsthisbook. Afundamentaldifferencefrommostdiscussionsofdiscrete-timesystemsistheapproachusingoperators.Operatorsmakeitpossibletoavoidtheconfusingnotionoftransform.Instead,theoperatorexpressionforadiscrete-timesystem,andthesystemsimpulse response are two representations for the same system; theyarethecoordinatesofapointasseenfromtwodifferentcoordinatesystems.Then a transformationofa system hasanactive meaning: for example,composingtwosystemstobuildanewsystem.


10/122

x Preface

HowtousethisbookAristotlewastutortotheyoungAlexanderofMacedon(later,theGreat).As ancient royalty knew, a skilled and knowledgeable tutor is the mosteffectiveteacher[3]. Askilledtutormakesfewstatementsandasksmanyquestions,forsheknowsthatquestioning,wondering,anddiscussingpromote long-lasting learning. Therefore, questions of two types are interspersedthroughthebook:questionsmarkedwitha inthemargin: Thesequestionsarewhatatutormightaskyouduringatutorial,andaskyoutoworkoutthenextstepsinananalysis. Theyareansweredinthesubsequenttext,whereyoucancheckyoursolutionsandmyanalysis.numbered

questions:

These

problems,

marked

with

a

shaded

background,

arewhata tutormightaskyou totakehomeaftera tutorial. Theygivefurtherpracticewiththetooloraskyoutoextendanexample,useseveraltoolstogether,orresolveparadoxes.Trylotsofquestionsofbothtypes!

CopyrightlicenseThisbookislicensedunderthe. . . license.AcknowledgmentsWegratefullythankthefollowingindividualsandorganizations:Forsuggestionsanddiscussions:. . . For thefree softwarefor typesetting: Donald Knuth (TEX); Han The Thanh(PDFTEX);HansHagenandTacoHoekwater(ConTEXt);JohnHobby(Meta-Post); AndyHammerlindl,JohnBowman,andTomPrince(asymptote);Richard

Stallman

(emacs);

the

Linux

and

Debian

projects.


11/122

1Differenceequations

1.1 Rabbits 21.2 Leakytank 71.3

Fall

of

afog

droplet

11

1.4 Springs 14

Theworld is toorichand complex for ourminds to grasp itwhole, forourmindsarebutasmallpartoftherichnessoftheworld. Tocopewiththecomplexity,wereasonhierarchically. Wedividetheworldintosmall,comprehensiblepieces:systems.Systemsareubiquitous:aCPU,amemorychips,amotor,awebserver,ajumbojet,thesolarsystem,thetelephonesystem,oracirculatorysystem. Systemsareausefulabstraction,chosen

becausetheirexternalinteractionsareweakerthantheirinternalinteractions.Thatpropertiesmakesindependentanalysismeaningful.Systemsinteractwithothersystemsviaforces,messages,oringeneralviainformationorsignals. Signalsandsystemsisthestudyofsystemsandtheirinteraction.Thisbook studies only discrete-time systems, where timejumps ratherthanchangescontinuously. Thisrestriction isnotassevereas itsseems.First,digitalcomputersare,bydesign,discrete-timedevices,sodiscrete-time

signals

and

systems

includes

digital

computers.

Second,

almost

all

theimportantideasindiscrete-timesystemsapplyequallytocontinuous-timesystems.Alas,evendiscrete-timesystemsaretoodiverseforonemethodofanalysis.Thereforeeventheabstractionofsystemsneedssubdivision.Theparticularclassofso-calledlinearandtime-invariantsystemsadmitspowerfultoolsofanalysisanddesign.Thebenefitofrestrictingourselvestosuch


12/122

2 1.1 Rabbits

systems,andthemeaningoftherestrictions,willbecomeclearinsubsequentchapters.

1.1 RabbitsHere isFibonaccisproblem [6, 10], a famousdiscrete-time, linear, time-invariantsystemandsignal:

Acertainmanputapairofrabbitsinaplacesurroundedonallsidesbyawall.How many pairs of rabbits canbe produced from that pair in a year if it issupposedthateverymontheachpairbegetsanewpairwhichfromthesecondmonthonbecomesproductive?

1.1.1 MathematicalrepresentationThissystemconsistsoftherabbitpairsandtherulesofrabbitreproduction.Thesignal is thesequence fwhere f[n] is the number of rabbit pairs atmonthn(theproblemasksaboutn= 12).Whatisfinthefirstfewmonths?Inmonth0, onerabbitpair immigrates into thesystem: f[0] = 1. Letsassumethattheimmigrantsarechildren.Thentheycannothavetheirownchildren

in

month

1

they

are

too

young

so

f[1] = 1.But

this

pair

is

an

adultpair,soinmonth2thepairhaschildren,makingf[2] = 2.Findingf[3] requiresconsideringtheadultandchildpairsseparately(hierarchicalreasoning),becauseeachtypebehavesaccordingtoitsownreproductionrule.Thechildpairfrommonth2growsintoadulthoodinmonth3,andtheadultpairfrommonth2begetsachildpair. Soinf[3] = 3: twoadultandonechildpair.The two adultpairs contribute two child pairs in month4, and the onechildpairgrowsup,contributinganadultpair.Somonth4hasfivepairs:two

child

and

three

adult

pairs.

To

formalize

this

reasoning

process,

define

twointermediatesignalscanda:

c[n] = numberofchildpairsatmonthn;a[n] = numberofadultpairsatmonthn.

Thetotalnumberofpairsatmonthnisf[n] = c[n] + a[n].Hereisatableshowingthethreesignalsc,a,andf:


13/122

31 Differenceequations

n 0 1 2 3c 1 0 1 1a 0 1 1 2f 1 1 2 3

Thearrowsinthetableshowhownewentriesareconstructed.Anupwarddiagonalarrowrepresentstheonlymeanstomakenewchildren,namelyfromlastmonthsadults:

a[n 1] c[n] or c[n] = a[n 1].Ahorizontalarrowrepresentsonecontributiontothismonthsadults,thatadultslastmonthremainadults:a[n 1]

a[n]. Adownwarddiagonal

arrow represents the other contribution to this months adults, that lastmonthschildrengrowupintoadults:c[n 1] a[n].Thesumofthetwocontributionsis

a[n] = a[n 1] + c[n 1].Whatisthedifferenceequationforfitself?Tofindtheequationforf,onehasatleasttwomethods:logicaldeduction(Problem1.1)ortrialanderror. Trialanderrorisbettersuitedforfindingresults,andlogicaldeductionisbettersuitedforverifyingthem.Therefore,usingtrialanderror,lookforapatternamongadditionsamplesoff:

n 0 1 2 3 4 5 6c 1 0 1 1 2 3 5a 0 1 1 2 3 5 8f 1 1 2 3 5 8 13

Whatusefulpatternsliveinthesedata?Oneprominentpatternisthatthesignalsc,a,andflooklikeshiftedversionsofeachother:

a[n] = f[n 1];c[n] = a[n 1] = f[n 2].

Sincef[n] = a[n] + c[n],


14/122

4 1.1 Rabbits

f[n] = f[n 1] + f[n 2].withinitialconditionsf[0] = f[1] = 1.Thismathematicaldescription, orrepresentation, clarifiesapoint that isnotobvious intheverbaldescription: thatthenumberofrabbitpairs inany month depends on the number in the two preceding months. Thisdifferenceequationissaidtobeasecond-orderdifferenceequation. Sinceits coefficientsare all unity, and thesigns arepositive, it is thesimplestsecond-orderdifferenceequation.Yetitsbehaviorisrichandcomplex.

Problem 1.1 VerifyingtheconjectureUsethetwointermediateequations

c[n] = a[n 1],a[n] = a[n 1] + c[n 1];

andthedefinitionf[n] = a[n] + c[n] toconfirmtheconjecturef[n] = f[n 1] + f[n 2].

1.1.2 Closed-formsolutionTherabbitdifferenceequationisanimplicitrecipethatcomputesnewvaluesfromoldvalues. Butdoesithaveaclosedform: anexplicitformulaforf[n] thatdependsonnbutnotonprecedingsamples? Asasteptoward finding a closed form, lets investigate how f[n] behaves as nbecomeslarge.Doesf[n] growlikeapolynomialinn,likealogarithmicfunctionofn,orlikeanexponentialfunctionofn?Decidingamongtheseoptionsrequiresmoredata. Herearemanyvaluesoff[n] (startingwithmonth0):

1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,...


15/122


Thesamplesgrowquickly.Theirgrowthistoorapidtobelogarithmic,unlessf[n]isanunusualfunctionlike(logn)20.Theirgrowth

is

probably

also

too

rapid

for

f[n]

tobe a polynomial in n, unless f[n] is na high-degree polynomial. A likely alternative isexponentialgrowth. To testthathypothesis,usepictorialreasoningbyplottinglnf[n]versusn. Theplottedpointsoscillateaboveandbelowabest-fitstraightline. Thereforelnf[n]growsalmostexactlylinearlywithn,andf[n]isapproximatelyanexponentialfunctionofn:

f[n]Azn,wherezandAareconstants.

lnf[n]

Howcanzbeestimatedfromf[n]data?n f[n]/f[n1]10 1.6181818181818

best-fitlineasngrows,theexponentialapprox-Becausetheplottedpointsfalleverclosertothe 20 1.6180339985218imation f[n] Aznbecomes more exact as n 30 1.6180339887505

40 1.6180339887499grows.Iftheapproximationwereexact,thenf[n]/f[n1]wouldalwaysequalz, so f[n]/f[n1]becomes 50 1.6180339887499anevercloserapproximationtozasnincreases.Theseratiosseemtoconvergetoz=1.6180339887499.Itsfirstfewdigits1.618mightbefamiliar. Foramemoryamplifier,feedtheratiototheonlineInverseSymbolicCalculator(ISC).Givenanumber,it guesses its mathematical source. When given the Fibonacci z, the InverseSymbolicCalculatorsuggeststwoequivalentforms: thatzisarootof1xx2 orthatitis (1+ 5)/2. Theconstantisthefamousgoldenratio[5].Therefore,f[n] An. Tofindtheconstantof

n f[n]/f[n1]proportionality A, takeout thebigpartbydividing f[n]by n. These ratios hover around0.723...,soperhapsAis31. Alas,exper 10

20 0.72362506926472

0.72360679895785

imentswithlargervaluesofnstronglysuggestthatthedigitscontinue0.723606... whereas3

3040

0.723606797750060.72360679774998

1 = 0.73205.... Abit of experimentation or 50 0.72360679774998the Inverse Symbolic Calculator suggests that0.72360679774998probablyoriginatesfrom/ 5.


16/122

6 1.1 Rabbits

Thisconjecturehasthemeritofreusingthe 5alreadycontainedinthedefinitionof,soitdoesnotintroduceanewarbitrarynumber. WiththatconjectureforA,theapproximationforf[n] becomes

n+1f[n] .5

Howaccurateisthisapproximation?Totesttheapproximation,takeoutthebig

n r[n]/r[n 1]partbycomputingtheresidual:

2 0.61803398874989601r[n] = f[n] n+1/ 5. 3 0.61803398874989812

4 0.61803398874988624The

residual

decays

rapidly,

perhaps

expo 5 0.61803398874993953nentially.Thenrhasthegeneralform 6 0.61803398874974236

r[n] Byn, 7 0.618033988750294148 0.61803398874847626

wherey and B are constants. To findy, 9 0.61803398875421256computetheratiosr[n]/r[n 1].Theycon 10 0.61803398873859083verge to 0.61803..., which isalmostex-

nactly1 or1/.Thereforer[n] B(1/) .WhatistheconstantofproportionalityB?

nTocomputeB,divider[n] by(1/) . Thesevalues,ifnisnottoolarge(Problem1.2), almost instantlysettleson0.27639320225. With luck, thisnumbercanbeexplainedusingand 5. Afewnumericalexperimentssuggesttheconjecture

1 1B= .

5 Theresidualbecomes

n+11 1r[n] = .

5 ThenumberofrabbitpairsisthesumoftheapproximationAzn andtheresidualByn:

f[n] = 1 n+1 ()(n+1) .5


17/122


Howbizarre! TheFibonaccisignalfsplitsintotwosignalsinatleasttwoways. First, it is thesumof theadult-pairssignal aand thechild-pairssignalc.Second,itisthesumf1 + f2 wheref1 andf2 aredefinedby

1f1[n] n+1;51

f2[n] (1/)n+1.5

Theequivalenceofthesedecompositionswouldhavebeendifficulttopredict. Instead, many experiments and guesses were needed to establishtheequivalence. Anotherkindofquestion,alsohardtoanswer,arisesbychangingmerelytheplussignintheFibonaccidifferenceequationintoaminussign:

g[n] = g[n 1] g[n 2].Withcorrespondinginitialconditions,namelyg[0] = g[1] = 1,thesignalgrunsasfollows(staringwithn= 0):

1,1,0,1,1,0,1,1,0,1,1, 0 , . . . . oneperiod

Ratherthangrowingapproximatelyexponentially,thissequenceisexactlyperiodic. Why? Furthermore,ithasperiod6. Why? Howcanthisperiod

bepredictedwithoutsimulation?Arepresentationsuitedforsuchquestionsisintroducedin??. Fornow,letscontinueinvestigatingdifferenceequationstorepresentsystems.

Problem 1.2 Actualresiduallnr[n]

nHereisasemiloggraphoftheabsoluteresidual|r[n]|computednumericallyupton=80. (Theabsoluteresidualisusedbecausethe residual is often negative and wouldhaveacomplexlogarithm.) Itfollowsthepredictedexponentialdecayforawhile,butthenmisbehaves.Why?

1.2 LeakytankIntheFibonaccisystem,therabbitschangedtheirbehaviorgrewuporhadchildrenonlyonceamonth.TheFibonaccisystemisadiscrete-time


18/122

8 1.2 Leakytank

system. Thesesystemsaredirectlysuitableforcomputationalsimulationandanalysisbecausedigitalcomputers themselvesact likediscrete-timesystems. However, manysystems in the world such as pianostrings,earthquakes,

microphones,

or

eardrums

are

naturally

described

as

continuous-timesystems.

Toanalyzecontinuous-timesystemsusingdiscrete-timetoolsrequiresapproximations.Theseapproximationsareillustratedinthesimplestinterestingcontinuous-timesystem: aleakytank. Imagineabathtuborsinkfilledtoaheighthwithwater. Attimet= 0,thedrainisopenedandwaterflowsout. Whatisthesubsequentheightofthewater?Att= 0,thewaterlevelandthereforethepressureisatitshighest,sothewaterdrainsmostrapidlyatt= 0.Asthewaterdrainsandthelevelfalls,thepressureandtherateofdrainagealsofall.Thisbehavioriscapturedbythefollowingdifferentialequation:

h

leak

dh= f(h),

dtwheref(h) isanas-yet-unknownfunctionoftheheight.Finding f(h) requires knowing the geometry of the tub and piping andthen

calculating

the

flow

resistance

in

the

drain

and

piping.

The

simplest

modelforresistanceisaso-calledlinearleak: thatf(h) isproportionaltoh.Thenthedifferentialequationsimplifiesto

dhh.dt

Whatarethedimensionsofthemissingconstantofproportionality?Thederivativeontheleftsidehasdimensionsofspeed(heightpertime),sothemissingconstanthasdimensionsofinversetime. Calltheconstant1/,whereisthetimeconstantofthesystem.Then

dh h= .

dt


19/122


Analmost-identicaldifferentialequationdescribesthevoltageVonacapacitordischargingacrossaresistor:

dV 1= V.

dt RC

R

C

V

Itistheleaky-tankdifferentialequationwithtimeconstant= RC.

Problem 1.3 KirchoffslawsUseKirchoffslawstoverifythisdifferentialequation.

Approximatingthecontinuous-timedifferentialequationasadiscrete-timesystemenables thesystem tobesimulatedbyhandandcomputer. Inadiscrete-timesystem,timeadvancesinlumps.Ifthelumpsize,alsoknownasthetimestep,isT,thenh[n] isthediscrete-timeapproximationofh(nT). Imaginethatthesystemstartswithh[0] = h0. Whatish[1]? Inotherwords,whatisthediscrete-timeapproximationforh(T)?Theleaky-tankequationsaysthat

dh h= .

dt Att= 0thisderivativeish0/. Ifdh/dtstaysfixedforawholetimestepaslightlydubiousbutsimpleassumptionthentheheightfallsbyapproximatelyh0T/ inonetimestep.Therefore

T Th[1] = h0 h0 = 1 h[0].

Usingthesameassumptions,whatish[2] and,ingeneral,h[n]?The reasoning to compute h[1] from h[0] applies when computing h[2]fromh[1]. Thederivativeatn = 1equivalently,att = T is h[1]/.Thereforebetweenn = 1andn = 2equivalently,betweent = T andt= 2Ttheheightfallsbyapproximatelyh[1]T/tau,

T Th[2] = h1 h1 = 1 h[1].

Thispatterngeneralizestoaruleforfindingeveryh[n]:

Th[n] = 1 h[n 1].


20/122

10 1.2 Leakytank

Thisimplicitequationhastheclosed-formsolutionn

Th[n] = h0 1 .

Howcloselydoesthissolutionreproducethebehavioroftheoriginal,continuous-timesystem?The original, continuous-time differential equation dh/dt = htau issolvedby

h(t) = h0et/.Atthediscretetimest= nT,thissolutionbecomes

nh(t) = h0enT/ = h0 eT/ .Thediscrete-timeapproximationreplaceseT/ with1 T/. ThatdifferenceisthefirsttwotermsintheTaylorseriesforeT/ : 2 3

eT/ = 1T +1 T 1 T + . . . . 2 6

Therefore the discrete-time approximation is accurate when the higher-ordertermsintheTaylorseriesaresmallnamely,whenT/

1.

Thismethodofsolvingdifferentialequationsbyreplacingthemwithdiscrete-timeanalogsisknownastheEulerapproximation,anditcanbeusedtosolveequationsthatareverydifficultorevenimpossibletosolveanalytically.

Problem 1.4 Whichistheapproximatesolution?

n

Here are unlabeled graphs showing the discrete-time samplesh[n] and thecontinuous-timesamplesh(nT), forn =0 . . . 6.Whichgraphshowsthediscrete-timesignal?Problem 1.5 LargetimestepsSketchthediscrete-timesamplesh[n] inthreecases:(a.)T =/2 (b.)T = (c.)T = 2 (d.)T = 3Problem 1.6 TinytimestepsShowthatasT0,thediscrete-timesolution


21/122


T nh[n] = h0 1 .

approachesthecontinuous-timesolution

h(t) = h0enT/.HowsmalldoesT havetobe,asafunctionofn,inorderthatthetwosolutionsapproximatelymatch?

1.3 FallofafogdropletTheleakytank(Section1.2)isafirst-ordersystem,anditsdifferentialequationanddifferenceequationarefirst-orderequations.However,thephysicalworldisoftensecondorderbecauseNewtonssecondlawofmotion,F= ma,containsasecondderivative.Forsuchsystems,howapplicableistheEulerapproximation?ToillustratetheissuesthatariseinapplyingtheEulerapproximationtosecond-ordersystems,letssimulatethefallofafogdropletactedonbygravity(F= mg)andairresistance. Afogdropletissmallenoughthatitsairresistanceisproportional to velocity rather than to the more usual velocity squared.Then the net downward force on the droplet is mg bv, wherev is itsvelocityandbisaconstantthatmeasuresthestrengthofthedrag.Intermsofpositionx,withthepositivedirectionasdownward,Newtonssecondlawbecomes

d2x dxm = mg b .

dt2 dtDividingbothsidesbymgives

d2x b dx= g .

dt2 m dtWhat

are

the

dimensions

of

b/m?

Theconstantb/mturnsthevelocitydx/dtintoanacceleration,sob/mhasdimensionsofinversetime.Thereforerewriteitas1/,wherem/bisatimeconstant.Then

d2x 1 dx= g .

dt2 dt


22/122

12 1.3 Fallofafogdroplet

Whatisadiscrete-timeapproximationforthesecondderivative?Intheleaky-tankequation,

dh h= ,dt thefirst derivative at t = nT had the Eulerapproximation (h[n+ 1] h[n])/Tandh(t= nT) becameh[n].Thesecondderivatived2x/dt2 isthelimitofadifferenceoftwofirstderivatives.UsingtheEulerapproximationprocedure,approximatethefirstderivativesatt= nTandt= (n+ 1)T:

dx

x[n+ 1] x[n] ;

dt t=nT Tdx x[n+ 2] x[n+ 1] .dt t=(n+1)T T

Thend2x 1x[n+ 2] x[n+ 1] x[n+ 1] x[n] .dt2 t=nT T T T

Thisapproximationsimplifiestod2x

1 (x[n+ 2] 2x[n+ 1] + x[n]) .dt2 t=nT T2

The Euler approximation for the continuous-time equation of motion isthen

1 1 x[n+ 1] x[n](x[n+ 2] 2x[n+ 1] + x[n]) = g

T2 Tor

Tx[n+ 2] 2x[n+ 1] + x[n] = gT2 (x[n+ 1] x[n]).

Ouroldfriendfromtheleakytank, theratioT/,hasreappearedinthisproblem. To simplify the subsequent equations, define T/. Thenaftercollecting the like terms, thedifferenceequation for the falling fogdropletis

x[n+ 2] = (2 )x[n+ 1] (1 )x[n] + gT2.


23/122


Asexpected, thisdifferenceequation issecondorder. Like theprevioussecond-orderequation,theFibonacciequation,itneedstwoinitialvalues.Letstryx[1] = x[0] = 0.Physically,thefogdropletstartsfromrestatthereference

height

0,

and

at

t= 0

starts

feeling

the

gravitational

force

mg.

Foratypicalfogdropletwithradius10m,thephysicalparametersare:

m4.21012kg;b2.8109kgs1;1.5103s1.

Relativeto,thetimestepTshouldbesmall,otherwisethesimulationerrorwilllarge.Atimestepsuch as 0.1ms is a reasonable compromisebe- x[n]

(m)

2010

tweenkeepingreducingtheerrorandspeeding 0upthesimulation. Thenthedimensionlessratio 0 10 20 nis0.0675.Asimulationusingtheseparametersshows x initially rising faster than linearly, probably quadratically, thenrisinglinearlyatarateofroughly1.5mpertimestepor1.5cms1.Thissimulationresultexplainsthelongevityoffog.Fogis,roughlyspeaking,acloudthathassunktotheground. Imaginethatthiscloudreachesupto500m(atypicalcloudthickness). Then,tosettletotheground,thecloudrequires

500mtfall 9hours.

1.5cms1Inotherwords,fogshouldlastovernightinagreementwithexperience!

Counting timestepsHowmanytimestepswould the fog-dropletsimulationrequire(withT = 0.1ms)inorderforthedroplettofall500minthesimulation?Howlongwouldyourcomputer,oranothereasilyavailablecomputer,requiretosimulatethatmanytimesteps?

Problem 1.7 TerminalvelocityBysimulatingthefogequation

x[n+ 2] = (2 )x[n+ 1] (1 )x[n] + gT2.withseveralvaluesofTandtherefore,guessarelationbetweeng,T,,andtheterminalvelocityoftheparticle.


24/122

14 1.4 Springs

1.4 SpringsNowletsextendoursimulationstothemostimportantsecond-ordersystem: thespring. Springsareamodelforavastnumberofsystemsinthenaturalandengineeredworlds: planetaryorbits,chemicalbonds,solids,electromagneticradiation,andevenelectronprotonbonds.Sincecolorresultsfromelectromagneticradiationmeetingelectronprotonbonds,grassisgreenandtheskyisbluebecauseofhowspringsinteractwithsprings.Thesimplestspringsystemisamassconnectedtoaspringandfreetooscillateinjustonedimension.Itsdifferentialequationis

d2xm + kx= 0,

dt2

wherexistheblocksdisplacementfromtheequilibriumposition,mistheblocksmass,andkisthespringconstant.Dividingbymgives

d2x k+ x= 0.

dt2 mDefiningtheangularfrequency k/mgivesthecleanequation:

d2x+ 2x= 0.

dt2NowdividetimeintouniformstepsofdurationT,andreplacethesecondderivatived2x/dt2 withadiscrete-timeapproximation:

d2x x[n+ 2] 2x[n+ 1] + x[n] ,dt2 t=nT T2

whereasusualthesamplex[n] correspondstothecontinuous-timesignalx(t) att= nT.Then

x[n+ 2] 2x[n+ 1] + x[n]+ 2x[n] = 0

T

2oraftercollectingliketerms,

x[n+ 2] = 2x[n+ 1] 1+ (T)2 x[n].DefiningT,

x[n+ 2] = 2x[n+ 1] 1+ 2 x[n].

km

x


25/122

151 Difference equations

This second-order difference equation needs two initial values. A simplepair is x[0] = x[1] = x0. This choice corresponds to pulling the mass right-wards by x0, then releasing it at t = T. What happens afterward?To simulate the system numerically,one shouldchoose T to make small. As a reasonable small, try 100 samples per oscillation period: = name: dummy

file: shm-forward2/100 or roughly 0.06. Alas, the simulation pre-

state: unknowndicts that the oscillations grow to infinity.What went wrong?Perhaps , even 0.06, is too large. Here are two simulations with smallerat values of:

x x

t t

0.031 0.016These oscillations also explode. The only difference seems to be the rate ofgrowth (Problem 1.8).

Problem 1.8 Tiny values of Simulate

x[n + 2] = 2x[n + 1] 1 + 2 x[n]using very small values for . What happens?

An alternative explanation is that the discrete-time approximation of thederivative caused the problem. If so, it would be surprising, because thesame approximation worked when simulating the fall of a fog droplet. Butlets try an alternative definition: Instead of defining

dx x[n + 1] x[n] ,dt t=nT T

try the simple change todx x[n] x[n 1] .dt T


26/122

Usingthesameprocedureforthesecondderivative,d2x

x[n] 2x[n 1] + x[n 2] .dt2 t=nT T2

Thediscrete-timespringequationisthen x(1+ 2)x[n] = 2x[n 1] x[n 2],

ort

2x[n 1] x[n 2]x[n] = .

1+ 2Usingthesameinitialconditionsx[0] = x[1] = 1,whatisthesubsequenttimecourse?Thebadnewsisthattheseoscillationsdecaytozero!However,thegoodnewsisthatchangingthede- xrivativeapproximationcansignificantlyaffectthe

behavior of thediscrete-time system. Lets try asymmetricsecondderivative: t

d2x x[n+ 1] 2x[n] + x[n 1] .dt2 t=nT T2

Thenthedifferenceequationbecomesx[n+ 2] = (2 2)x[n+ 1] x[n].

Nowthesystemoscillatesstably,justasaspringwithoutenergy lossorinputshouldbehave.Why did the simple change to a symmetric second derivative solve theproblem of decaying or growing oscillations? The representation of thealternativediscrete-timesystemsasdifferenceequationsdoesnothelpanswerthatquestion. Itsanswerrequiresthetwomost important ideas insignalsandsystems:operators(??)andmodes(??).

Problem 1.9 Differentinitialconditionsx

t

Herearethesubsequentsamplesusingthesymmetricsecondderivativeandinitialconditionsx[0] = 0,x[1] = x0. Theamplitude is, however, much largerthan x0. Is thatbehavior physically reasonable? Ifyes,explainwhy.Ifnot,explainwhatshouldhappen.


27/122

2Differenceequationsandmodularit

2.1 Modularity:Makingtheinputliketheoutput 172.2 Endowmentgift 212.3

Rabbits

25

Thegoalsofthischapterare: toillustratemodularityandtodescribesystemsinamodularway; totranslateproblemsfromtheirrepresentationasaverbaldescrip

tionintotheirrepresentationasdiscrete-timemathematics(differenceequations);and

tostartinvestigatingthesimplestsecond-ordersystem,thesecond-simplestmoduleforanalyzinganddesigningsystems.

Thethemesofthischapteraremodularityandtherepresentationofverbaldescriptionsasdiscrete-timemathematics. Weillustratethesethemeswith twoexamples, money inahypotheticalMITendowment fundandrabbitsreproducinginapen,settingupdifferenceequationstorepresentthem. Therabbitexample,whichintroducesanewmoduleforbuildingandanalyzingsystems,isafrequentvisitortothesechapters.Inthischapterwebegintostudyhowthatmodulebehaves. Before introducingtheexamples,weillustratewhatmodularityisandwhyitisuseful.

2.1 Modularity:MakingtheinputliketheoutputAcommonbutalasnon-modularwaytoformulatedifferenceanddifferentialequationsusesboundaryconditions. Anexamplefrompopulation


28/122


29/122

192 Differenceequationsandmodularity

meaningofphilosophicalasirrelevantbutansweringithelpsustounderstandanddesignsystems. HerethesystemistheUnitedStates. Theinputtothesystemisonenumber,theinitialpopulationp[2007];however,the

output

is

a

sequence

of

populations

p[2008], p[2009], . . ..

In

this

formulation,thesystemsoutputcannotbecometheinputtoanothersystem.

Therefore we cannot design large systemsby combining small, easy-tounderstand systems. Nor wecan we analyze large, hard-to-understandsystemsbybreakingthemintosmallsystems.Instead, wewould likeamodular formulation inwhich the input is thesame kind of object as the output. Here is the US-population questionreformulatedalongthoselines:Ifx[n] peopleimmigrateintotheUnitedstatesinyearn,andtheUSpopulationgrowsat1%annually,whatisthepopulationin

yearn

?The

input

signal

is

the

number

of

immigrants

versus

time,

so

it

is

asequenceliketheoutputsignal. Includingtheeffectofimmigration,therecurrenceis

p[n] = (1+ r)p[n 1] + x[n] . output reproduction immigration

Theboundaryconditionisnolongerseparatefromtheequation! Insteaditispartoftheinputsignal.Thismodularformulationisnotonlyelegant;itisalsomoregeneralthanistheformulationwithboundaryconditions,forwecanrecasttheoriginalquestionintothisframework. Therecastinginvolvesfindinganinputsignalheretheimmigrationversustimethatreproducestheeffectoftheboundaryconditionp[2007] = 3108.

Pausetotry 2. What input signal reproduces the effect of theboundarycondition?

Theboundaryconditioncanbereproducedwiththisimmigrationschedule(theinputsignal):

3108 ifn= 2007;x[n] = 0 otherwise.

ThismodelimaginesanemptyUnitedStatesintowhich300millionpeoplearriveintheyear2007. Thepeoplegrow(innumbers!) atanannualrate


30/122

20 2.1Modularity:Makingtheinputliketheoutput

of1%,andwewanttoknowp[2077],theoutputsignal(thepopulation)intheyear2077.Thegeneralformulationwithanarbitraryinputsignalishardertosolvedirectlythanisthefamiliarformulationusingboundaryconditions,whichcanbesolvedbytricksandguesses.Forourinputsignal,theoutputsignalis

31081.01n2007 forn2007;p[n] = 0 otherwise.

Exercise 2. Checkthatthisoutputsignalsatisfiestheboundaryconditionandthepopulationequation.

In later chapters you learn how to solve the formulation with an arbitraryinputsignal. Hereweemphasizenotthemethodofsolutionbutthemodularformulationwhereasystemturnsonesignalintoanothersignal.Thismodulardescriptionusingsignalsandsystemshelpsanalyzecomplexproblemsandbuildcomplexsystems.Toseehowithelps,firstimagineaworldwithtwocountries:IrelandandtheUnitedStates.SupposethatpeopleemigratefromIrelandtotheUnitedStates,areasonablemodelinthe1850s. SupposealsothattheIrishpopulationhasanintrinsic10annualdeclineduetofaminesandthatanother10%ofthepopulationemigrateannuallytotheUnitedStates.IrelandandtheUnitedStatesaretwosystems,withonesystemsoutput(Irishemigration)feeding intotheothersystems input(theUnitedStatess immigration).Themodulardescriptionhelpswhenprogrammingsimulations.Indeed,giantpopulation-growthsimulationsareprogrammedinthisobject-orientedway. Eachsystemisanobjectthatknowshowitbehaveswhatitoutputswhen fed inputsignals. Theuserselectssystemsandspecifiesconnectionsamongthem. Fluid-dynamicssimulationsuseasimilarapproach

by

dividing

the

fluid

into

zillions

of

volume

elements.

Each

elementisasystem,andenergy,entropy,andmomentumemigratebetween

neighboringelements.Our one- or two-component population systems are simpler than fluid-dynamicssimulations, thebetter to illustratemodularity. Using twoexamples,wenextpracticemodulardescriptionandhowtorepresentverbaldescriptionsasmathematics.


31/122

2 Differenceequationsandmodularity 21

2.2 EndowmentgiftThefirstexampleforrepresentingdescriptionsasmathematicsinvolvesahypotheticalendowmentgifttoMIT.Adonorgives107 dollarstoMITto support projects proposed and chosenby MIT undergraduates! MITwould like touse this fund fora long timeanddraw 0.5106 everyyearforaso-called5%drawdown. Assumethatthemoneyisplacedinareliableaccountearning4%interestcompoundedannually.HowlongcanMITanditsundergraduatesdrawonthefundbeforeitdwindlestozero?Nevermakeacalculationuntilyouknowroughlywhattheanswerwillbe! ThismaximisrecommendedbyJohnWheeler,abrilliantphysicistwhosemostfamousstudentwasMITalumRichardFeynman [9]. WehighlyrecommendWheelersmaximasawaytobuildintuition.Sohereareafewestimationquestionstogetthementaljuicesflowing. Startwiththebroadestdistinction,whetheranumberisfiniteorinfinite.Thisdistinctionsuggeststhefollowingquestion:

Pausetotry 3. Willthefundlastforever?

Alas, the fund will not last forever. In the first year, the drawdown isslightlygreaterthantheinterest,sotheendowmentcapitalwilldwindleslightly. Asaresult,thenextyearsinterestwillbesmallerthanthefirstyearsinterest. Sincethedrawdownstaysthesameat$500,000annually(whichis5%oftheinitialamount),thecapitalwilldwindlestillmoreinlateryears,reducingtheinterest,leadingtoagreaterreductionininterest,leadingtoagreaterreductionincapital. . . Eventuallythefundevaporates.Giventhatthe lifetime isfinite,roughlyhowlongisit? Canyourgreat-grandchildrenuseit?

Pauseto

try

4.

Will

the

fund

last

longer

than

or

shorter

than

100

years?

The figure of 100 years comes from the differencebetween the outflow theannualdrawdownof5%of thegiftand the inflowproducedbythe interest rate of 4%. The differencebetween 5% and 4% annually is


32/122

22 2.2 Endowmentgift

= 0.01/year. Thedimensionsofareinversetime,suggestinganendowmentlifetimeof1/,whichis100years. Indeed, ifeveryyearwerelikethefirst,thefundwouldlastfor100years. However,theinflowfrominterest

decreases

as

the

capital

decreases,

so

the

gap

between

outflow

and

inflow increases. Thus this1/method,basedonextrapolating thefirstyearschangetoeveryyear,overestimatesthelifetime.Havingwarmedupwithtwoestimates, letsdescribethesystemmathematicallyandsolveforthetrue lifetime. Indoingso,wehavetodecidewhatistheinputsignal,whatistheoutputsignal,andwhatisthesystem.Thesystemistheleasttrickypart:Itisthebankaccountpaying4interest.Thegiftof$10millionismostlikelypartoftheinputsignal.

Pausetotry 5. Isthe$500,000annualdrawdownpartoftheoutputortheinputsignal?

Thedrawdownflowsoutof theaccount, and theaccount is thesystem,soperhapsthedrawdown ispartoftheoutputsignal. No!! Theoutputsignaliswhatthesystemdoes,whichistoproduceoratleasttocomputeabalance.Theinputsignaliswhatyoudotothesystem.Here,youmovemoneyinoroutofthesystem:

bankaccountmoneyinorout balance

Theinitialendowmentisaone-timepositiveinputsignal,andtheannualdrawdownisarecurringnegativeinputsignal. Tofindhowlongtheendowment lasts, find when the output signal crossesbelow zero. Theseissuesofrepresentationarehelpfultofigureoutbeforesettingupmathematics.Otherwisewithgreateffortyoucreateirrelevantequations,whereuponnoamountofcomputingpowercanhelpyou.Now lets represent the description mathematically. First represent theinputsignal. Tominimize the largenumbers and dollar signs, measuremoneyinunitsof$500,000.Thischoicemakestheinputsignaldimensionless:

X=20,1,1,1,1,...


33/122


Weusethenotationthatacapitalletterrepresentstheentiresignal,whilealowercaseletterwithanindexrepresentsonesamplefromthesignal. Forexample, P is thesequenceofpopulationsandp[n] is thepopulation inyear

n.

Theoutputsignalis

Y= 20,?,?,?, . . . Pausetotry 6. Explainwhyy[0] = 20.

Theproblemistofillinthequestionmarksintheoutputsignalandfindwhenitfallsbelowzero.Thedifferenceequationdescribingthesystemis

y[n] = (1+ r)y[n 1] + x[n],whereristheannualinterestrate(here,r= 0.04).Thisdifferenceequationisafirst-orderequationbecauseanyoutputsampley[n] dependsontheoneprecedingsampley[n 1].Thesystemthattheequationrepresentsissaidtobeafirst-ordersystem. Itisthesimplestmoduleforbuildingandanalyzingcomplexsystems.

Exercise 3. ComparethisequationtotheoneforestimatingtheUSpopulationin2077.Nowwehaveformulatedtheendowmentproblemasasignalprocessed

byasystem toproduceanothersignalallhailmodularity! andrepresentedthisdescriptionmathematically. However,wedonotyetknowhowtosolvethemathematicsforanarbitraryinputsignalX.Buthereweneedtosolveitonlyfortheparticularinputsignal

X= 20,

1,

1,

1,

1,

. . . .

Withthatinputsignal,therecurrencebecomes

y[n] = 1.04y[n 1] 1 n > 0;20 n= 0.

They[0] = 20reflectsthat thedonorseeds theaccountwith20unitsofmoney, which is the $10,000,000 endowment. The 1 in the recurrence


34/122

24 2.2 Endowmentgift

reflectsthatwedraw1uniteveryyear. Withoutthe1term,thesolutiontotherecurrencewouldbey[n] 1.04n,wherethesymbolmeansexceptfor a constant. The 1 means that simple exponential growth is not asolution.

However,

1

is

a

constant

so

it

may

contribute

only

a

constant

to

thesolution. Thatreasoningisdubiousbutsimple,sotryitfirst. Usinga

bitofcourage,hereisaguessfortheformofthesolution:y[n] = A1.04n + B (guess),

whereAandBareconstantstobedetermined. BeforefindingAandB,figureoutthemostimportantcharacteristic,theirsigns.So:

Pausetotry 7. Assumethatthisformiscorrect.WhatarethesignsofAandB?

Sincetheendowmenteventuallyvanishes,thevariabletermA 1.04nmustmakeanegativecontribution; soA < 0. Since the initialoutputy[0] ispositive,BmustovercomethenegativecontributionfromA; so B > 0.

Pausetotry 8. FindAandB.SolvingfortwounknownsAandBrequirestwoequations.Eachequationwillprobablycomefromonecondition. Somatchtheguesstotheknown

balancesattwotimes.Thetimes(valuesofn)thatinvolvetheleastcalculationaretheextremecasesn = 0andn = 1. Matchingtheguesstothe

behavioratn= 0givesthefirstequation:20= A+ B (n= 0condition).

Tomatch theguess to thebehaviorat n = 1, firstfindy[1]. At n = 1,whichisoneyearafterthegift,0.8unitsofinterestarrivefrom4%of20,and1unitleavesasthefirstdrawdown.So

y[1] = 20+ 0.8 1= 19.8.Matchingthisvaluetotheguessgivesthesecondequation:

19.8= 1.04A+ B (n= 1condition).


35/122


Bothconditionsaresatisfiedwhen A = 5and B = 25. Aspredicted,A < 0 andB > 0.Withthatsolutiontheguessbecomes

y[n] = 25 5

1.04n.Thissolutionhasastrangebehavior. Afterthebalancedropsbelowzero,the1.04n growsevermorerapidlysothebalancebecomesnegativeeverfaster.

Exercise 4. Doesthatbehaviorofbecomingnegativemoreandmore rapidly indicate an incorrect solution to therecurrencerelation,oranincompletemathematicaltranslationofwhathappensinreality?

Exercise 5. Theguess,withthegivenvaluesforA andB,worksforn = 0andn = 1. (Howdoyouknow?) Showthatitisalsocorrectforn > 1.

Nowwecananswertheoriginalquestion: Whendoesy[n] falltozero?nAnswer: When1.04 > 5,whichhappensatn = 41.035.... SoMITcan

drawon the fund inyears1 , 2 , 3 , . . . , 4 1, leaving loosechange in theaccountforalargegraduationparty.Theexactcalculationisconsistentwiththeargumentthatthelifetimebelessthan100years.

Exercise 6. HowmuchloosechangeremainsafterMITdrawsitslastpayment?Converttorealmoney!

2.3 RabbitsThesecondsystemtorepresentmathematicallyisthefecundityofrabbits.The Encyclopedia Britannica (1981 edition) states this population-growthproblemasfollows[6]:


36/122


37/122


Letc[n] bethenumberofchildpairsatmonthnanda[n] bethenumberofadultpairsatmonthn.Theseintermediatesignalscombinetomaketheoutputsignal:

f[n] = a[n] + c[n] (outputsignal).Pausetotry 11. What equation contains the rule that childrenbe

comeadultsinonemonth?

Becausechildrenbecomeadultsinonemonth,andadultsdonotdie,thepoolofadultsgrowsbythenumberofchildpairsinthepreviousmonth:

a[n] = a[n 1] + c[n 1] (growing-upequation).The two terms on the right-hand side represent the two ways tobe anadult:1. Youwereanadultlastmonth(a[n 1]),or2. youwereachildlastmonth(c[n 1])andgrewup.Thenextequationsaysthatalladults,andonlyadults,reproducetomakenewchildren:

c[n] = a[n 1].However, this equation is not completebecause immigration also contributes child pairs. The number of immigrant pairs at month n is theinputsignalx[n].Sothefullstoryis:

c[n] = a[n 1] + x[n] (childequation)Ourgoalisarecurrenceforf[n],thetotalnumberofpairs.Soweeliminatethenumberofadultpairsa[n] andthenumberofchildpairsc[n] infavoroff[n]. Doitintwosteps. First,usethegrowing-upequationtoreplacea[n 1] inthechildequationwitha[n 2] + c[n 2]. Thatsubstitutiongives

c[n] = a[n 2] + c[n 2] + x[n].


38/122

28 2.3 Rabbits

Sincef[n] = c[n] + a[n],wecanturntheleftsideintof[n] byaddinga[n].Thegrowing-upequationsaysthata[n] isalsoa[n 1] + c[n 1],soaddthosetermstotherightsideandprayforsimplification.Theresultis

c[n] + a[n] = a[n 2] + c[n 2] +x[n] + a[n 1] + c[n 1] . f[n] f[n2] f[n1]

Theleftsideisf[n]. Therightsidecontainsa[n 2] + c[n 2],whichisf[n 2];anda[n 1] + c[n 1],whichisf[n 1].Sothesumofequationssimplifiesto

f[n] = f[n 1] + f[n 2] + x[n].The Latin problem description is from Fibonaccis LiberAbaci [10], published in1202, and thisequation is the famousFibonaccirecurrencebutwithaninputsignalx[n] insteadofboundaryconditions.Thismathematicalrepresentationclarifiesonepointthatisnotobviousintheverbalrepresentation: Thenumberofpairsofrabbitsatmonthndependsonthenumberinmonthsn1 andn2.Becauseofthisdependenceontwoprecedingsamples,thisdifferenceequationisasecond-orderdifferenceequation.Sinceallthecoefficientsareunity,itisthesimplestequation of that category, and ideal as a second-order system to understandthoroughly.Tobuildthatunderstanding,weplaywiththesystemandseehowitresponds.

2.3.2 TryingtherecurrenceToplaywiththesystemdescribedbyFibonacci,weneedtorepresentFi

bonaccisboundaryconditionthatonepairofchildrabbitsenterthewallsonlyinmonth0. ThecorrespondinginputsignalisX=1 , 0 , 0 , 0 , . . .. UsingthatX,knownasanimpulseoraunitsample,therecurrenceproduces(leavingouttermsthatarezero):

f[0] = x[0] = 1,f[1] = f[0] = 1,f[2] = f[0] + f[1] = 2,f[3] = f[1] + f[2] = 3,. . .


39/122

2 Differenceequationsandmodularity 29

Whenyoutryafewmorelines,yougetthesequence:F=1,1,2,3,5,8,13,21,34,..Whenyou tireofhandcalculation, askacomputer tocontinue. Here isslowPythoncodetoprintf[0],f[1],. . .,f[19]:def f(n):

i f n < 2 : return 1return f(n-1) + f(n-2)

print [f(i) for i in range(20)]Exercise 7. Write the corresponding Matlab or Octave code,

then rewrite the code in one of the languages Python,Matlab,orOctavetobeefficient.

Exercise 8. WriteMatlab, Octave, orPythoncode tofindf[n]whentheinputsignalis1 , 1 , 1 , . . ..Whatisf[17]?

2.3.3 RateofgrowthTosolvetherecurrenceinclosedformmeaninganexplicitformulaforf[n]thatdoesnotdependonprecedingsamplesitishelpfultoinvestigateitsapproximategrowth.Evenwithoutsophisticatedtechniquestofindtheoutputsignal,wecanunderstandthegrowthinthiscasewhentheinputsignalistheimpulse.

Pausetotry 12. Whentheinputsignalistheimpulse,howfastdoesf[n]grow? Is itpolynomial, logarithmic,orexponential?

Fromlookingatthefirstfewdozenvalues,itlookslikethesequencegrowsquickly. Thegrowth isalmostcertainly toorapidtobe logarithmicand,almostascertain,toofasttobepolynomialunlessitisahigh-degreepolynomial.Exponentialgrowthisthemostlikelycandidate,meaningthatanapproximationforf[n]is


40/122

30 2.3 Rabbits

f[n] znwherez isaconstant.Toestimatez,playwiththerecurrencewhenn > 0,whichiswhentheinputsignaliszero. Thef[n] areallpositiveand,sincef[n] = f[n 1] + f[n 2] whenn > 0,thesamplesareincreasing: f[n] >f[n 1].Thisboundturnsf[n] = f[n 1] + f[n 2] intotheinequality

f[n] < f[n 1] + f[n 1].Sof[n] < 2f[n 1] orf[n]/f[n 1] < 2;thereforetheupperboundonzisz < 2.Thisboundhasacounterpartlowerboundobtainedbyreplacingf[n 1] byf[n 2] in theFibonaccirecurrence. Thatsubstitutionturnsf[n] = f[n 1] + f[n 2] into

f[n] > f[n 2] + f[n 2].Therightsideis2f[n 2] sof[n] > 2f[n 2].Thisboundleadstoalower

bound:z2 > 2 orz > 2.Therangeofpossiblez isthen

2 < z < 2. Letschecktheboundsbyexperiment.Hereisthesequenceofratiosf[n]/f[n1] forn = 1 , 2 , 3 , . . .:

1.0, 2.0, 1.5, 1.666 . . . , 1.6, 1.625, 1.615 . . . , 1.619 . . . , 1.617 . . .

Theratiosseemtooscillatearound1.618,whichliesbetweenthepredictedbounds 2and2. Inlaterchapters,usingnewmathematicalrepresentations,youlearnhowtofindtheclosedfromforf[n].Wehavewalkedtwostepsinthatdirectionbyrepresentingthesystemmathematicallyandbyinvestigatinghowf[n] grows.

Exercise 9. Useamorerefinedargumenttoimprovetheupperboundtoz <

3.

Exercise 10. Doesthenumber1.618lookfamiliar?


41/122


Exercise 11. [Hard!]Considerthesamesystembutwithonerabbit pair emigrating into the system every month,not only in month 0. Compare the growth withFibonaccisproblem, whereonepairemigrated inmonth0only. Isitnowfasterthanexponential? Ifyes,howfastisit? Ifno,doestheorderofgrowthchangefromz1.618?


42/122


43/122

3Blockdiagramsandoperators:Twonewrepresentations

3.1

Disadvantagesof

difference

equations

34

3.2 Blockdiagramstotherescue 353.3 Thepowerofabstraction 403.4 Operationsonwholesignals 413.5 Feedbackconnections 453.6 Summary 49

Thegoalsofthischapterare: tointroducetworepresentationsfordiscrete-timesystems:block

diagramsandoperators; tointroducethewhole-signalabstractionandtoexhortyoutouse

abstraction; tostartmanipulatingoperatorexpressions; tocompareoperatorwithdifference-equationandblock-diagram

manipulations.

Thepreceding chaptersexplained the verbal-description anddifference-equation representations. This chapter continues the theme of multiplerepresentationsbyintroducingtwonewrepresentations:blockdiagramsand operators. New representations are valuablebecause they suggestnewthoughtsandoftenprovidenewinsight;anexpertengineervaluesherrepresentationsthewayanexpertcarpentervalueshertools.Thischapterfirstintroducesblockdiagrams,discussesthewhole-signalabstractionand


44/122

34 3.1 Disadvantagesofdifferenceequations

thegeneralvalueofabstraction,thenintroducestheoperatorrepresentation.

3.1 DisadvantagesofdifferenceequationsChapter2illustratedthevirtuesofdifferenceequations. Whencomparedtotheverbaldescriptionfromwhichtheyoriginate,differenceequationsarecompact,easytoanalyze,andsuitedtocomputerimplementation.Yetanalyzingdifferenceequationsofteninvolveschainsofmicro-manipulationsfromwhichinsightishardtofind.Asanexample,showthatthedifferenceequation

d[n] = a[n] 3a[n 1] + 3a[n 2] a[n 3]isequivalenttothissetofequations:

d[n] = c[n] c[n 1]c[n] = b[n] b[n 1]b[n] = a[n] a[n 1].

Asthefirststep,usethelastequationtoeliminateb[n] andb[n 1] fromthec[n] equation:

c[n] = (a[n] a[n 1]) (a[n 1] a[n 2]) = a[n]2a[n1]+a[n2]. b[n] b[n1]Usethatresulttoeliminatec[n] andc[n 1] fromthed[n] equation:

d[n] = (a[n] 2a[n 1] + a[n 2]) (a[n 1] 2a[n 2] + a[n 3]) c[n] c[n1]

= a[n] 3a[n 1] + 3a[n 2] a[n 3].Voil:Thethree-equationsystemisequivalenttothesingledifferenceequation.

But

what

a

mess.

Each

step

is

plausible

yet

the

chain

of

steps

seems

random.Ifthelaststephadproduced

d[n] = a[n] 2a[n 1] + 2a[n 2] a[n 3],it would not immediately look wrong. We would like a representationwhereitwouldlookwrong,perhapsnotimmediatelybutatleastquickly.Blockdiagramsareonesuchrepresentation.


45/122

353 Blockdiagramsandoperators:Twonewrepresentations

Exercise 12. Althoughthissectionpointedoutadisadvantageofdifferenceequations,itisalsoimportanttoappreciatetheirvirtues.Therefore,inventaverbaldescription(astory)torepresentthesingleequation

d[n] = a[n] 3a[n 1] + 3a[n 2] a[n 3]and then a verbal description to represent theequivalent set of three equations. Now have funshowing, without converting to difference equations,thatthesedescriptionsareequivalent!

3.2 BlockdiagramstotherescueBlockdiagramsvisuallyrepresentasystem.Toshowhowtheywork,hereareafewdifferenceequationswithcorrespondingblockdiagrams:

Delay1/2+ y[n] = (x[n] + x[n 1])/2

averagingfilter+

Delayy[n] = y[n 1] + x[n]

accountwith0%interest

Pausetotry 13. Draw theblock diagram for the endowment accountfromSection2.2.

Theendowmentaccountisabankaccountthatpays4%interest,soitneedsagainelement in the loop, withgainequal to 1.04. Thediagram isnotunique. Youcanplacethegainelementbeforeorafterthedelay. Hereisonechoice:


46/122

36 3.2 Blockdiagramstotherescue

+

1.04Delay

y[n] = 1.04y[n 1] + x[n]endowmentaccountfromSection2.2

Amazingly,allsystemsinthiscoursecanbebuiltfromonlytwoactionsandonecombinator:

A action1:multiplybyDelay

+action2:delayonetickcombinator:addinputs

3.2.1 BlockdiagramfortheFibonaccisystemTopracticeblockdiagrams,wetranslate(represent)theFibonaccisystemintoablockdiagram.

Pausetotry 14. RepresenttheFibonaccisystemofSection1.1usingablockdiagram.

WecouldtranslateFibonaccisdescription(Section1.1)directlyintoablockdiagram,butweworkedsohardtranslatingthedescriptionintoadifferenceequationthatwestartthere.Itsdifferenceequationis

f[n] = f[n 1] + f[n 2] + x[n],where the inputsignalx[n]ishowmanypairsofchildrabbitsenterthesystematmonthn,andtheoutputsignalf[n] ishowmanypairsofrabbitsareinthesystematmonthn. Intheblockdiagram,itisconvenienttoletinputsignalsflowinfromtheleftandtoletoutputsignalsexitattherightfollowingtheleft-to-rightreadingcommontomanylanguages.


47/122


Exercise 13. Do signals-and-systems textbooks in Hebrew orArabic,whicharewrittenrighttoleft,putinputsignalsontherightandoutputsignalsontheleft?

TheFibonaccisystemcombinestheinputsample,thepreviousoutputsample,andthesecond-previousoutputsample.Thesethreesignalsarethereforeinputstothepluselement. Thepreviousoutputsampleisproducedusingadelayelementtostoresamplesforonetimetick(onemonth)beforesendingthemonward.Thesecond-previousoutputsampleisproducedbyusingtwodelayelementsinseries. SotheblockdiagramoftheFibonaccisystemis

+ DelayDelayDelay

f[n]x[n]

3.2.2 ShowingequivalenceusingblockdiagramsWe introducedblock diagrams in the hope of finding insight not easilyvisiblefromdifferenceequations.Souseblockdiagramstoredotheproofthat

d[n] = a[n] 3a[n 1] + 3a[n 2] a[n 3]isequivalentto

d[n] = c[n] c[n 1],c[n] = b[n] b[n 1],b[n] = a[n] a[n 1].

Thesystemofequationsisacascadeofthreeequationswiththestructureoutput= thisinput previousinput.

Theblockdiagramforthatstructureis

-1 Delay+


48/122

38 3.2 Blockdiagramstotherescue

wherethegainof1producesthesubtraction.Thecascadeofthreesuchstructureshastheblockdiagram

Delay+

-1 Delay+

-1 Delay-1

Thisdiagramhasadvantagescomparedtothesetofdifferenceequations.First,thediagramhelpsusdescribethesystemcompactly. Eachstageinthecascade isstructurally identical, and thestructural identity isapparentbylookingatit.Whereasinthedifference-equationrepresentation,thecommonstructureofthethreeequationsishiddenbythevaryingsignalnames.

Each

stage,

it

turns

out,

is

a

discrete-time

differentiator,

the

simplestdiscrete-timeanalogofacontinuous-timedifferentiator.Sotheblock

diagrammakesapparentthatthecascadeisadiscrete-timetripledifferentiator.Second,theblockdiagramhelpsrewritethesystem,whichweneedtodotoshowthat it is identical to thesingledifferenceequation. Sofollowasignalthroughthecascade. Thesignalreachesaforkthreetimes(markedwithadot),andeachforkoffersachoiceofthebottomortopbranch.Threetwo-waybranchesmeans23 or8pathsthroughthesystem.Letsexaminea

few

of

them.

Three

paths

accumulate

two

delays:

1. lowroad,lowroad,highroad:

Delay+

-1 Delay+

-1 Del-1

2. lowroad,highroad,lowroad:

-1 Delay+

-1 Delay+

-1 Del

3. highroad,lowroad,lowroad:


49/122

3 Blockdiagramsandoperators:Twonewrepresentations 39

-1 Delay+

-1 Delay+

-1 Del

Besidesthetwodelays,eachpathaccumulatestwogainsof1,makingagainof1.Sothesumofthethreepathsisagainof3andadoubledelay.

Exercise 14. Show the other five pathsare: threepaths withasingledelayandagainof1,onepathwiththreedelays and a gain of 1, and one path that goesstraightthrough(nogain,nodelay).

Ablockdiagramrepresentingthosefourgroupsofpathsis

3 Delay

3 Delay Delay

1 Delay Delay Delay

+

Thesingledifferenceequationd[n] = a[n] 3a[n 1] + 3a[n 2] a[n 3].

alsohasthisblockdiagram.Thepictorialapproachisanadvantageofblockdiagramsbecausehumansare sensorybeings and vision is an important sense. Brains, over hundreds of millions of years of evolution, have developed extensive hardwaretoprocesssensoryinformation. However,analyticalreasoningandsymbolmanipulationoriginatewithlanguage,skillperhaps100,000yearsold,soourbrainshavemuch lesspowerfulhardware in thosedomains.


50/122

40 3.3 Thepowerofabstraction

Notsurprisingly,computersarefarmoreskilledthanarehumansatanalytical tasks likesymbolic algebra and integration, and humans are farmoreskilledthanarecomputersatperceptualtaskslikerecognizingfacesor

speech.

When

you

solve

problems,

amplify

your

intelligence

with

a

visualrepresentationsuchasblockdiagrams.

Ontheotherside,exceptbytracingandcountingpaths,wedonotknowtomanipulateblockdiagrams;whereasanalyticrepresentationslendthemselves to transformation, an important property when redesigning systems. So we need a grammar forblock diagrams. To find the rules ofthisgrammar,weintroduceanewrepresentationforsystems,theoperatorrepresentation. Thisrepresentationrequiresthewhole-signalabstractionin which all samples of a signal combine into one signal. It is a subtlechange

of

perspective,

so

we

first

discuss

the

value

of

abstraction

in

general,thenreturntotheabstraction.

3.3 ThepowerofabstractionAbstraction is a great tools of human thought. All language isbuilt onit: Whenyouuseaword,youinvokeanabstraction. Theword,evenanordinarynoun,standsforarich,subtle,complexidea. Takecowandtrytoprogramacomputertodistinguishcowsfromnon-cows;thenyoufindhowdifficultabstractionis. Orwatchachildsabilitywithlanguagedevelopuntilshelearnsthatredisnotapropertyofaparticularobjectbutisanabstractpropertyofobjects. Nooneknowshowthemindmanagestheseamazingfeats,norinwhatamountstothesameignorancecananyoneteachthemtoacomputer.Abstraction is so subtle that even Einstein once missed its value. Einstein formulated the theory of special relativity [7] with space and timeasseparateconceptsthatmingleintheLorentztransformation.Twoyearslater, the mathematician Hermann Minkowskijoined the two ideas intothespacetimeabstraction:

TheviewsofspaceandtimewhichIwishtolaybeforeyouhavesprungfromthesoilofexperimentalphysics,andthereinliestheirstrength.Theyareradical.Henceforth spaceby itself, and timeby itself, are doomed to fade away intomereshadows,andonlyakindofunionofthetwowillpreserveanindependentreality.


51/122


See the English translation in [11] or the wonderful textbook SpacetimePhysics [1], whosefirstauthorrecentlyretired from theMITphysicsdepartment. Einsteinthoughtthatspacetimewasapreposterousinventionof

mathematicians

with

time

to

kill.

Einstein

made

a

mistake.

It

is

perhaps the fundamentalabstraction ofmodernphysics. Themoral is that

abstractionispowerfulbutsubtle.

Exercise 15. Findafewabstractionsinchemistry,biology,physics,andprogramming.

IfwelackEinsteinsphysicalinsight,weoughtnottocompoundtheabsence

with

his

mistake.

So

look

for

and

create

abstractions.

For

example,

inaprogram,factoroutcommoncodeintoaprocedureandencapsulatecommonoperationsintoaclass. Ingeneral,organizeknowledgeintoabstractionsorchunks[15].

3.4 OperationsonwholesignalsForsignalsandsystems,thewhole-signalabstractionincreasesourabilitytoanalyzeandbuildsystems. Theabstractionistakeallsamplesofasignalandlumpthemtogether,operatingontheentiresignalatonceandasoneobject.Wehavenotbeenthinkingthatwaybecausemostofourrepresentationshinderthisview. Verbaldescriptionsanddifferenceequationsusuallyimplyasample-by-sampleanalysis.Forexample,fortheFibonaccirecurrenceinSection2.3.2,wefoundthezerothsamplef[0],usedf[0]tofindf[1],usedf[0]andf[1]tofindf[2],foundafewmoresamples,thengottiredandaskedacomputertocarryon.Blockdiagrams,thethirdrepresentation,seemtoimplyasample-by-sampleanalysisbecausethedelayelementholdsontosamples,spittingoutthesampleafteronetimetick.Butblockdiagramsliveinbothworldsandcanalsorepresentoperationsonwholesignals.Justreinterprettheelementsinthewhole-signalview,asfollows:


52/122

42 3.4 Operationsonwholesignals

A action1:multiplywholesignalbyDelay

+action

2:

shift

whole

signal

right

one

tick

combinator:addwholesignals

Tobenefitfromtheabstraction, compactlyrepresent theprecedingthreeelements. Whenasignalisasingleobject,thegainelementactslikeordinarymultiplication,andthepluselementactslikeadditionofnumbers.Ifthedelayelementcouldalsoactlikeanarithmeticoperation,thenallthreeelementswouldact ina familiarway, andblockdiagramscouldbemanipulatedusingtheordinaryrulesofalgebra. Inordertobringthedelayelementintothisfamiliarframework,weintroducetheoperatorrepresentation.

3.4.1 OperatorrepresentationInoperatornotation, thesymbolRstandsfortheright-shiftoperator. Ittakesasignalandshiftsitonesteptotheright. HereisthenotationforasystemthatdelaysasignalX byoneticktoproduceasignalY:

Y=R{X}.Nowforgetthecurlybraces,tosimplifythenotationandtostrengthentheparallelwithordinarymultiplication.Thecleannotationis

Y=RX.Pausetotry 15. Convinceyourselfthatright-shiftoperatorR,rather

thantheleft-shiftoperatorL,isequivalenttoadelay.

LetstesttheeffectofapplyingRtothefundamentalsignal,theimpulse.Theimpulseis

I=1 , 0 , 0 , 0 , . . . ApplyingRtoitgives


53/122


54/122

44 3.4 Operationsonwholesignals

Withthesemappings,thedifferenceequationturnsintothecompactformD= (1 3R+ 3R2 R3)A.

Toshowthatthisformisequivalenttothesystemofthreedifferenceequations,translatethemintoanoperatorexpressionconnectingtheinputsignalAandtheoutputsignalD.

Pausetotry 16. Whataretheoperatorversionsofthethreedifferenceequations?

Thesystemofequationsturnsintotheseoperatorexpressionsd[n] = c[n] c[n 1] D= (1 R)C,c[n] = b[n] b[n 1] C= (1 R)B,b[n] = a[n] a[n 1] B= (1 R)A.

EliminateBandCtogetD= (1 R)(1 R)(1 R)A= (1 R)3A.

ExpandingtheproductgivesD= (1 3R+ 3R2 R3)A,

which matches the operator expression corresponding to the single difference equation. The operator derivation of the equivalence is simplerthantheblock-diagramrewriting,andmuchsimplerthanthedifference-equationmanipulation.Nowextendtheabstractionbydividingouttheinputsignal:

D= 1 3R+ 3R2 R3.

ATheoperatorexpressionontheright,beingindependentoftheinputandoutputsignals,isacharacteristicofthesystemaloneandiscalledthesystemfunctional.The moral of the example is that operators help you efficiently analyzesystems. Theyprovideagrammarforcombining,forsubdividing,andingeneral forrewritingsystems. It isa familiargrammar, thegrammarof


55/122


algebraicexpressions.Letsseehowextensivelyoperatorsfollowthese.Inthenextsectionwestretchtheanalogyandfindthatitdoesnotbreak.

Exercise 16. What is theresultofapplying1 Rto thesignal1, 2, 3, 4, 5, . . .?

Exercise 17. Whatistheresultofapplying(1 R)2 tothesignal1,4,9,16,25,36,...?

3.5 FeedbackconnectionsThesystemwith(1 R)3 asitssystemfunctionalusedonlyfeedforwardconnections:Theoutputcouldbecomputeddirectlyfromafixednumberofinputs. However,manysystemssuchasFibonacciorbankaccountscontainfeedback,wheretheoutputdependsonpreviousvaluesoftheoutput. Feedback produces new kinds of system functionals. Lets testwhethertheyalsoobeytherulesofalgebra.

3.5.1 AccumulatorHereisthedifferenceequationforthesimplestfeedbacksystem,anaccumulator:

y[n] = y[n 1] + x[n].Itisabankaccountthatpaysnointerest. Theoutputsignal(thebalance)isthesumoftheinputs(thedeposits,whetherpositiveornegative)uptoandincludingthattime.Thesystemhasthisblockdiagram:

+

Delay

NowcombinethevisualvirtuesofblockdiagramswiththecompactnessandsymbolicvirtuesofoperatorsbyusingRinsteadofDelay.Theoperatorblockdiagramis


56/122

46 3.5 Feedbackconnections

X +

R

Y

Pausetotry 17. Whatisitssystemfunctional?Either from this diagram or from the difference equation, translate intooperatornotation:

Y= RY+ X.CollecttheYtermsononeside,andyoufindendupwiththesystemfunctional:

Y 1= .

X 1 RItisthereciprocalofthedifferentiator.ThisoperatorexpressionisthefirsttoincludeRinthedenominator. Onewaytointerpretdivisionistocomparetheoutputsignalproducedbythedifferenceequationwiththeoutputsignalproducedbythesystemfunctional

1/(1 R).

For

simplicity,

test

the

equivalence

using

the

impulse

I= 1 , 0 , 0 , 0 , . . .

asthe inputsignal. Sox[n]is1forn = 0and is0otherwise. Thenthedifferenceequation

y[n] = y[n 1] + x[n]producestheoutputsignal

Y= 1 , 1 , 1 , 1 , . . . .

Exercise 18. Checkthisclaim.

Theoutputsignalisthediscrete-timestepfunction.Nowapply1/(1R)to the impulseIby importing techniques fromalgebraorcalculus. Use


57/122

47

1


syntheticdivision,Taylorseries,orthebinomialtheoremtorewrite1/(1 R)as

=1

+R+R2 +R3 + .

1 R

Toapply1/(1 R)totheimpulse,applytheeachofterms1,R,R2,. . . totheimpulseI:

1I=1, 0, 0, 0, 0, 0, 0, . . . , RI=0, 1, 0, 0, 0, 0, 0, . . . ,

R2I=0, 0, 1, 0, 0, 0, 0, . . . , R3I=0, 0, 0, 1, 0, 0, 0, . . . , R

4

I=0, 0, 0, 0, 1, 0, 0, . . . , . . .

AddthesesignalstogettheoutputsignalY.

Pausetotry 18. WhatisY?

Forn0,they[n]samplegetsa1 fromtheRnI term,andfromnootherterm.

So

the

output

signal

is

all

1s

from

n

=

0 onwards.

The

signal

with

thosesamplesisthestepfunction:

Y=1 , 1 , 1 , 1 , . . . . Fortunately,thisoutputsignalmatchestheoutputsignalfromrunningthedifferenceequation.So,foranimpulseinputsignal,theseoperatorexpressionsareequivalent:

1and 1 +R+R2+R3 + .

1 RExercise 19. Ifyouaremathematicallyinclined,convinceyour

selfthatverifyingtheequivalenceforthe impulseissufficient. Inotherwords,wedonotneedtotryallotherinputsignals.


58/122

48 3.5 Feedbackconnections

ThemoralisthattheRoperatorfollowstherulesofalgebraandcalculus.Sohavecourage:Useoperatorstofindresults,anddonotworry.

3.5.2 FibonacciTakingourownadvice,wenowanalyzetheFibonaccisystemusingoperators.Therecurrenceis:

output=delayedoutput + twice-delayedoutput + input.

Pausetotry 19. Turnthisexpressionintoasystemfunctional.

TheoutputsignalisF,andtheinputsignalisX.ThedelayedoutputisRX,andthetwice-delayedoutputisRRXorR2X. So

F=RF+R2F+X.CollectallFtermsononeside:

FRFR2F=X.ThenfactortheF:

(1RR2)F=X.ThendividebothsidesbytheRexpression:

1F= X.

1RR2Sothesystemfunctionalis

1.

1RR2


59/122


60/122


61/122

4Modes

4.1 GrowthoftheFibonacciseries 524.2 TakingoutthebigpartfromFibonacci 554.3 Operatorinterpretation 574.4 Generalmethod:Partialfractions 59

Thegoalsofthischapterare: to illustrate the experimental way that an engineer studies sys

tems,evenabstract,mathematicalsystems; toillustratewhatmodesarebyfindingthemfortheFibonaccisys

tem;and to decompose second-order systems into modes, explaining the

decompositionusingoperatorsandblockdiagrams.Thefirstquestioniswhatamodeis.Thatquestionwillbeansweredaswedecompose theFibonaccisequence intosimplersequences. Eachsimplesequencecanbegeneratedbyafirst-ordersystemliketheleakytankandiscalledamodeofthesystem.BydecomposingtheFibonaccisequenceintomodes,wedecomposethesystemintosimpler,first-ordersubsystems.The plan of the chapter is to treat the Fibonacci system first as ablack

boxproducinganoutput signal Fand to developcomputationalprobestoexaminesignals. Thisexperimentalapproachishowanengineerstudiesevenabstract,mathematicalsystems. Theresultsfromtheprobeswillshowushow todecompose thesignal into itsmodes. Thesemodesarethenreconciledwithwhattheoperatormethodpredictsfordecomposingthesystem.


62/122

52 4.1 GrowthoftheFibonacciseries

Whydescribetheexperimental,andperhapsharder,methodforfindingthemodesbeforegivingtheshortcutsusingoperators? WeknowtheoperatorexpressionfortheFibonaccisystem,andcouldjustrewriteitusingalgebra.

The

answer

is

that

the

operator

method

has

meaning

only

after

you feelmodes inyourfingertips, a feelingdevelopedonlyasyouplaywithsignals. Withoutfirstplaying, wewouldbe teachingyouamazingfeatsofcalculationonmeaninglessobjects.Furthermore,theexperimentalapproachworksevenwhennodifferenceequationisavailabletogeneratethesequence. Engineersoftencharacterizesuchunknownorpartiallyknownsystems.Thesystemmightbe: computational: Imaginedebuggingsomeoneelsesprogram. Yousend

intestinputstofindouthowitworksandwhatmakesitfail. electronic: ImaginedebuggingaCPUthatjustreturnedfromthefabri

cationrun,perhapsinquantitiesofmillions,butthatdoesnotcorrectlydividefloating-pointnumbers[12]. Youmightgiveitnumberstodivide until you find the simplest examples that give wrong answers.Fromthatdatayoucanoftendeducetheflawinthewiring.

mathematical:Imaginecomputingprimestoinvestigatethetwin-primeconjecture[16],oneoftheoutstandingunsolvedproblemsofnumbertheory.[Theconjecturestatesthatthereareaninfinitenumberofprimepairs

p,p+2

,such

as

(3,5),(5,7)

,etc.]

The

new

field

of

experimental

mathematics,whichusescomputationaltoolstoinvestigatemathematicsproblems,islively,growing,andafertilefieldforskilledengineers[4,14,8].

Sowehopethat,throughexperimentalprobesoftheFibonaccisequence,youlearnageneralapproachtosolvingproblems.

4.1 GrowthoftheFibonacciseriesSection1.1.2estimatedhowfastthesequencef[n]grew.Itseemedtogrowgeometricallywithanorderofgrowthbetween 2and2.Ourfirstprojectistoexperimentallynarrowthisrangeandtherebytoguessaclosedformfortheorderofgrowth.Oneprobetofindtheorderofgrowthistocomputethesuccessiveratiosf[n]/f[n1]. Theratiososcillatedaround1.618,butthisestimate isnot


63/122

4Modes 53

accurateenoughtoguessaclosedform. Sincetheoscillationsintheratiodieoutasngrows,letsestimatetheratioaccuratelybycomputingitforlargen.Ourtoolfortheseexperimentsourprobeisacomputerthatweprogram

in

Python,

a

clean,

widely

available

language.

Use

any

tool

that

fitsyou,perhapsanotherlanguage,agraphingcalculator,oraspreadsheet.Section2.3.2offeredthisPythoncodetocomputef[n]:def f(n):

i f n < 2 : return 1return f(n-1) + f(n-2)

Butthecodeisslowwhennislarge.Herearetherunningtimestoevaluatef[n]onaPentiumCoreDuo1.8GHzprocessor:

n 10 15 20 25 30 time(ms) 0.17 1.5 21 162 1164

Thetimesgrowrapidly!

Exercise 21. Whatistherunningtimeofthisimplementation?

Thetimesmightseem lowenoughtobeusable,but imaginebeingonadesert

island

with

only

a

graphing

calculator;

then

the

times

might

be

a

factorof10orof100longer. Wewouldliketobuildanefficientcomputationalprobesothatitiswidelyusable.Anefficientfunctionwouldstorepreviouslycomputedanswers,returningthestoredanswerwhenpossibleratherthanrecomputingoldvalues. InPython,onecanstorethevaluesinadictionary,whichisanalogoustoahashinPerloranassociativearrayinawk. ThememoizedversionoftheFibonaccifunctionis:

memo = {}def f(n):

if n < 2 : return 1if n in memo : return memo[n]

memo[n] = f(n-1) + f(n-2)return memo[n]


64/122

54 4.1 GrowthoftheFibonacciseries

Pausetotry 20. Whatistherunningtimeofthememoizedfunction,inbig-Ohnotation?

Thenewfunctionrunsinlineartimeafasterprobe!sowecaninexpensivelycomputef[n]forlargen.Herearetheratiosf[n]/f[n1]:

n f[n]/f[n1]5 1.60000000000000009

10 1.6181818181818181715 1.6180327868852459920 1.6180339985218033025 1.6180339886704431230

1.61803398875054083

35 1.6180339887498895740 1.6180339887498949045 1.61803398874989490

Thesevaluesareverystablebyn =45,perhapslimitedinstabilityonlybytheprecisionofthefloating-pointnumbers.Letsseewhatclosedformwouldproducetheratio1.61803398874989490atn =45. Onesourceforclosedformsisyourintuitionandexperience.AnotherwonderfulsourceistheInverseSymbolicCalculatorByusingtheInverseSymbolicCalculator,youincreaseyourrepertoireofclosedformandtherebyenhanceyourintuition.

Pausetotry 21. AsktheInverseSymbolicCalculatorabout1.6180339887498949TheInverseSymbolicCalculatorthinksthat1.61803398874989490ismostlikelythepositiverootofx2 x1or,equivalently,isthegoldenratio:

1+ 5

2

Letsusethathypothesis.Thenf[n]n.

Butwedonotknowtheconstanthiddenbythesymbol. FindthatconstantbyusingtheInverseSymbolicCalculatoronemoretime. Hereisa

.
http://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.htmlhttp://oldweb.cecm.sfu.ca/projects/ISC/ISCmain.html


65/122

4Modes 55

table of the ratio f[n]/n. With luck it converges to a constant. And itdoes:

n f[n]/n0 1.00000000000000000

10 0.7236250692647178120 0.7236067989578528530 0.7236067977500580940 0.7236067977499780550 0.7236067977499778360 0.7236067977499774970 0.7236067977499772780 0.7236067977499770590 0.72360679774997672

100 0.72360679774997649 Aroundn= 10,theratioslooklike 3 10.732butlaterratiosstabilizearoundavalueinconsistentwiththatguess.

Pausetotry 22. AsktheInverseSymbolicCalculatorabout0.7236067977499764Whichofthealternativesseemmostreasonable?

TheInverseSymbolicCalculatorprovidesmanyclosedformsfor0.723606797749976

Achoicethatcontains 5isreasonablesincecontains 5. Theclosed formnearestto0.72360679774997649andcontaining 5is(1+ 1/ 5)/2,whichisalso/ 5.SotheFibonaccisequenceisroughly

f[n] n.

54.2 TakingoutthebigpartfromFibonacci

Nowletstakeoutthebigpartbypeelingawaythe n contributionto5

seewhatremains.DefinethesignalF1by

f1[n] = n.5

ThissignalisonemodeoftheFibonaccisequence.Theshapeofamodeisitsorderofgrowth,whichhereis.Theamplitudeofamodeistheprefactor,whichhereis/ 5.Themodeshapeisacharacteristicofthesystem,


66/122

56 4.2 TakingoutthebigpartfromFibonacci

whereastheamplitudedependsontheinputsignal(forthisexample,theinputsignalwastheimpulse).Soweoftenhavemoreinterestintheshapethan in theamplitude. However, hereweneedshapeandamplitude inorder

to

determine

the

signal

and

peel

it

away.

SotabulatetheresidualsignalF2 = F F1:

n f2[n] = f[n] f1[n]0 +0.276393202250021061 0.170820393249936812 +0.105572809000084263 0.065247584249852774 +0.040325224750231045 0.024922359499623076

+0.01540286525060708

7 0.009519494249015998 +0.005883371001587539 0.00363612324743201

10 +0.00224724775415552Theresidualsignalstartssmallandgetssmaller,sothemainmodeF1 isanexcellentapproximationtotheFibonaccisequenceF.TofindaclosedformfortheresidualsignalF2,retrythesuccessive-ratiosprobe:

n f2[n]/f2[n 1]1 0.618033988749894462 0.618033988749896013 0.618033988749893904 0.618033988749890465 0.618033988749939536 0.618033988749742367 0.618033988750294148 0.618033988748476269 0.61803398875421256

10 0.61803398873859083Thesuccessiveratiosarealmostconstantandlooksuspiciouslylike1 ,whichisalso1/.

Exercise 22. Showthat1 = 1/.


67/122

4Modes 57

Sof2[n] ()n. Toevaluatetheamplitude,dividef2[n]bythemodeshape()n.Hereisatableofthoseresults:

n f2[n]/()n1 0.276393202250020902 0.276393202250021403 0.276393202250021014 0.276393202250019015 0.276393202250038996 0.276393202249970837 0.276393202250149418 0.276393202249514979 0.27639320225144598

10 0.27639320224639063Thosevaluesstabilizequicklyandlooklikeoneminustheamplitudeofthen mode.Sotheamplitudeofthe()n modeis1/ 5,whichisalso1/( 5).Thustheresidualsignal,combiningitsshapeandamplitude,is

1f2[n] = ()n.

5NowcombinetheF1 andF2 signalstogettheFibonaccisignal:

f[n] = f1[n] + f2[n] 1

= n + ()n.5 5Thisclosedform,deducedusingexperiment,isthefamousBinetformulaforthenth Fibonaccinumber.

Exercise 23. Usepeelingawayandeducatedguessingtofindaclosedformfortheoutputsignalwhentheimpulseisfedintothefollowingdifferenceequation:

y[n] = 7y[n1] 12y[n2] + x[n].

4.3 OperatorinterpretationNextwe interpretthisexperimentalresultusingoperatorsandblockdiagrams. Modes are the simplest persistent responses that a system can


68/122

58 4.3 Operatorinterpretation

make,andarethebuildingblocksofallsystems,sowewouldliketofindtheoperatororblock-diagramrepresentationsforamode.The Fibonacci signal decomposed into two simpler signals F1 and F2 whicharealsothemodesandeachmodegrowsgeometrically. Geometricgrowthresultsfromonefeedbackloop.Sothen modeisproducedbythissystem

+

R

withthe

system

functional

(1

R)

1.

The()n modeisproducedbythissystem+

1R

withthesystemfunctional(1+ R/)1.TheFibonaccisystemisthesumofthesesignalsscaledbytherespectiveamplitudes,soitsblockdiagramisaweightedsumoftheprecedingblockdiagrams. Thesystem functionalfortheFibonaccisystem isaweightedsumofthepure-modesystemfunctionals.Soletsaddtheindividualsystemfunctionalsandseewhatturnsup:

F(R) = F1(R) + F2(R) 1 1 1

= + 51 R 51+ R/

1= .

1 R R2ThatfunctionalisthesystemfunctionalfortheFibonaccisystemderiveddirectlyfromtheblockdiagram(Section3.5.2)! Sotheexperimentalandoperatorapproachesagreethattheseoperatorblockdiagramsareequivalent:


69/122

4Modes 59

15

11+R/

11RR2

5

11R

+=

where,tomakethediagrameasiertoparse,systemfunctionalsstandforthefirst- andsecond-ordersystemsthattheyrepresent.

Exercise 24. Write the system of difference equations that corresponds to the parallel-decompositionblock diagram. Show that the system is equivalent to theusualdifferenceequation

f[n] = f[n 1] + f[n 2] + x[n].Theequivalenceisobviousneitherfromtheblockdiagramsnorfromthedifferenceequationsdirectly.Makingtheequivalenceobviousneedseitherexperimentortheoperatorrepresentation.Havingexperimented,youarereadytousetheoperatorrepresentationgenerallytofindmodes.

4.4 Generalmethod:PartialfractionsSowewouldlikeawaytodecomposeasystemwithoutpeelingawayandguessing.Andwehaveone:themethodofpartialfractions,whichshowsthevalueof theoperatorrepresentationandsystemfunctional. Becausethesystemfunctionalbehaveslikeanalgebraicexpressionoronemightsay,becauseitisanalgebraicexpressionitisofteneasiertomanipulatethanistheblockdiagramorthedifferenceequation.Havinggonefromthedecomposedfirst-ordersystemstotheoriginalsecond-ordersystemfunctional,letsnowgotheotherway:fromtheoriginalsystem functional to the decomposed systems. To do so, first factor the Rexpression:

1 1 1= .

1 R R2 1 R 1+ R/


70/122

60 4.4 Generalmethod:Partialfractions

Thisfactoring,aseriesdecomposition,willhelpusstudypolesandzerosina later chapter. Here we use it to find theparallel decompositionbyusingthetechniqueofpartialfractions.Thepartialfractionsshouldusethetwofactorsindenominator,soguessthisform:

1 a b= + ,

1 R R2 1 R 1+ R/where a and b are unknown constants. After adding the fractions, thedenominatorwillbetheproduct(1 R)(1+ R/) andthenumeratorwillbetheresultofcrossmultiplying:

a(1+ R/) + b(1 R) = a+ (a/)R+ b bR.Wewantthenumeratortobe1. If we set a= andb= 1/,thenatleasttheRtermscancel,leavingonlytheconstanta+b. Sowechoseaandbtoolargebythesuma+ b,whichis+ 1/or 5.Soinsteadchoose

a= / 5,b= 1/( 5).

Ifyouprefersolvinglinearequationstotheguess-and-checkmethod,herearethelinearequations:

a+ b= 1,a/ b= 0,

whosesolutionsaretheonesdeducedusingtheguess-and-checkmethod.Themoral:Tofindhowasystembehaves,factoritssystemfunctionalandusepartialfractionstodecomposethatfactoredformintoasumoffirst-ordersystems.Withthatdecomposition,youcanpredicttheoutputsignal

becauseyouknowhowfirst-ordersystemsbehave.Youcanpracticethenewskillofdecompositionwiththefollowingquestion:


71/122

4Modes 61

Exercise 25. Lookagainatthesystemy[n] = 7y[n 1] 12y[n 2] + x[n].

Decomposetheoperatorrepresentationintoasumof two modes and draw the correspondingblockdiagram (usingblock diagram elements). Whenthe input signal X is the impulse, do the operatorandblock-diagramdecompositionsproducethesame closed form that you findby peeling awayandguessing?


72/122


73/122

5Repeatedroots

5.1 Leaky-tankbackground 645.2 Numericalcomputation 655.3

Analyzing

the

output

signal

67

5.4 Deformingthesystem:Thecontinuityargument 685.5 Higher-ordercascades 70

Afterreadingthischapteryoushouldbeable touseacontinuityargument toexplainthenon-geometricoutputofamodewithadoubleroot.

Modesgeneratepersistentoutputs. Sofarourexamplesgeneratepersistentgeometricsequences. Butamodefromarepeatedroot,suchasfromthesystemfunctional(1R/2)3,producesoutputsthatarenotgeometricsequences. Howdoesrootrepetitionproduce thisseeminglystrange

behavior?Theanalysisdependsontheideathatrepeatedrootsareanunlikely,specialsituation. Ifrootsscatterrandomlyonthecomplexplane, theprobability iszero that tworoots landexactlyon thesameplace. Ageneric,decentsystemdoesnothaverepeatedroots,andonlythroughspecialcontrivancedoesaphysicalsystemacquirerepeatedroots. Thisfactsuggestsdeformingarepeated-rootsystemintoagenericsystembyslightlymovingonerootsothatthemodesofthedeformedsystemproducegeometricsequences. Thisnewsystemisthereforequalitativelyeasiertoanalyzethanistheoriginalsystem,anditcanapproximatetheoriginalsystemascloselyasdesired. Thiscontinuityargumentdependsontheideathattheworld


74/122


75/122

655 RepeatedrootsVout R

= .Vin 1 (1 )R

Exercise 27. Whatblock diagram corresponds to this systemfunctional?

Adoublerootarisesfromcascadingtwoidenticalsystems.Hereisitshigh-levelblockdiagramshowingtheinput,intermediate,andoutputsignals:

leakytankorRCcircuit leakytankorRCcircuitV0

V1V2

Itssystemfunctionalisthesquareofthefunctionalforonesystem: 2V2 R

= .V0 1 (1 )R

Thenumerator(R)2 doesnotaddinterestingfeaturestotheanalysis,sosimplifylifebyignoringit.Tosimplifythealgebrafurther,define= 1.WiththatdefinitionandwithouttheboringRfactor,thepurifiedsystemis

V2 1= .V0 (1 R)2

5.2 NumericalcomputationBydesign,thiscascadesystemhasadoublerootat= 1 .Letssimulateitsimpulseresponseandfindpatternsinthedata.Simulationrequireschoosingnumericalvaluesfortheparameters,andheretheonlyparameteris= T/.AnaccuratediscretizationusesatimestepTmuchshorterthanthesystemtimeconstant;otherwisethesystemchangessignificantlybetweensamples,obviatingthediscrete-timeapproximation.Souse1.

Pausetotry 24. Writeaprogramtosimulatetheimpulseresponse,choosingareasonableor.


76/122

66 5.2 Numericalcomputation

The following simulation uses = 0.05 or = 0.95 in computing theimpulseresponse:from scipy import *N = 100impulse = zeros(N)impulse[0] = 1beta = 0.95# return the output signal from feeding INPUT signal througha system# with a feedback loop containing a delay and the given GAIN.def onestage(input, gain):output = input * 0

output[0] = input[0]for i in range(1,len(output)): # 1..n-1

output[i] = input[i] + gain*output[i-1]return output

signal = impulse # start with the impulsefor gain in [beta, beta]: # run it through each system

signal = onestage(signal, gain)print signalTheimpulseresponseis:

n y[n]0 1.0000001 1.9000002 2.7075003 3.4295004 4.0725315 4.6426866 5.1456437 5.5866988 5.9707849 6.302494

10 6.586106. . .


77/122

5 Repeatedroots 67

5.3 AnalyzingtheoutputsignalThe impulse response contains a pattern. To find it, play with the dataandmakeconjectures.Thefirstfewsampleslookliken+ 1.However,byn= 10thatconjecturelooksdubious.Solookforanotherpattern.Asinglesystem(1 R)1wouldhaveageometric-sequenceoutputwherethenthsampleisn. Maybethatgeometricdecayappearsinthedoublesystemandswampstheconjecturedn+ 1growth.Therefore,takeoutthebigpartfromtheimpulseresponsebytabulatingthesignaly[n]/0.95n. Todoso,addonelineofcodetothepreviousprogram:print signal/0.95**arange(N)Thedataare

n y[n]/0.95n0 1.0000001 2.0000002 3.0000003 4.0000004 5.0000005 6.0000006 7.0000007 8.0000008 9.0000009 10.000000

10 11.000000Nowy[n] = n+ 1isexact!Theimpulseresponseofthedoublecascadeisthesignal

y[n] = (n+ 1) 0.95n forn0.The factor of 0.95n makes sensebecause a single system (1 0.95R)1wouldhave0.95nasitsimpulseresponse.Buthowdoesthefactorofn+ 1arise? To understand its origin, one method is convolution, which wasdiscussedinthelecture.Hereweshowanalternativemethodusingacontinuityargument.


78/122

68 5.4 Deformingthesystem:Thecontinuityargument

5.4 Deformingthesystem:ThecontinuityargumentThe cascade is hard to analyzebecause its roots are replicated. So deformthecascadebymakingthesecondrootbe0.951insteadof0.95.Thatslightlydeformedsystemhasthefunctional

1 1 .1 0.951R 1 0.95R

Sincetheroothardlymoved,theimpulseresponseshouldbealmostthesameastheimpulseresponseoftheoriginalsystem. Thisassumptionisthe essence of the continuity argument. We could find the responsebyslightlymodifyingtheprecedingprogram. However,reachingforaprogramtoooftendoesnotaddinsight.Alternatively,nowthatthesystemsrootsareunequal,wecaneasilyusepartialfractions.Thefirststepinpartialfractionsistofindthemodes:

1 1M1 = and M2 = .

1 0.951R 1 0.95RThesystemfunctionalisalinearcombinationofthesemodes:

1 1 C1 C2 = .1 0.951R 1 0.95R 1 0.951R 1 0.95R

Exercise 28. ShowthatC1 = 951andC2 = 950.Thepartial-fractionsdecompositionis

1 1 1 0.951 0.95 = .1 0.95R 1 0.951R 0.001 1 0.951R 1 0.95R

The0.951/(1 0.951R) systemcontributestheimpulseresponse0.951n+1,andthe0.95/(1 0.95R) systemcontributestheimpulseresponse0.95n+1.

Exercise 29. Checktheseimpulseresponses.

Sotheimpulseresponseofthedeformedsystemisy[n] = 1000(0.951n+1 0.95n+1).


79/122

695 Repeatedroots

Since 0.951 0.95, the difference in parentheses is tiny. However, thedifferenceismagnifiedbythefactorof1000outsidetheparentheses. Theresultingsignalisnottiny,andmightcontainthenon-geometricfactorofn

+ 1

in

the

impulse

response

of

atrue

double

root.

Toapproximatethedifference0.951n+1 0.95n+1,usethebinomialtheorem,keepingonlythetwolargestterms:

0.951n+1 = (0.95+ 0.001)n+1 0.95n+1 + (n+ 1)0.95n 0.001+ .

Thustheapproximateimpulseresponseisy[n] 1000 (n+ 1) 0.95n 0.001.

Thefactorof1000cancelsthefactorof0.001toleavey[n] (n+ 1) 0.95n,

whichiswhatweconjecturednumerically!Thusthelinearprefactorn+ 1comesfromsubtractingtwogarden-variety,geometric-sequencemodesthatarealmost identical. Thesignreflectsthatwekeptonlythefirsttwotermsinthebinomialexpansionof0.951n+1.However, as the deformation shrinks, the shifted root at 0.951becomesinstead0.9501or0.95001,etc. Astherootapproaches0.95,thebinomialapproximationbecomesexact,asdoestheimpulseresponse(n+ 1) 0.95n.Theresponse(n+ 1) 0.95n istheproductofanincreasingfunctionwithadecreasingfunction,witheachfunctionfightingforvictory. Insuchsituations,onefunctionusuallywinsatthen 0extreme, and theotherfunctionwinsatthen extreme,withamaximumproductwherethetwofunctionsarrangeadraw.

Exercise 30. Sketchn+ 1,0.95n,andtheirproduct.

Pausetotry 25. Whereisthemaximumof(n+ 1) 0.95n?Thisproductreachesamaximumwhentwosuccessivesamp

course note signals

Documents