7/28/2019 SciAm 1983 12 Hayes FSA

1/6

s .A. D ~ c 8 3 COMPUTER RECREATIONS

On the finite-state machine, a minimal modelo fmousetraps, ribosomes and the human soul

by Brian Hayes

T he most powerful computers haveneither hardware no r software;they are built ou t of pure thoughtstuff. Among these abstract machine sthe most ce lebrated is the one inventedin 1936 by th e British mathematicianAlan Mathison Turing. It ca n do moretha n a ny co mputer made of mere siliconeve r could; indeed, it ca n compute a n y ~thing th a i can be computed. A related class of conceptual computers lackthe omnipotence of the Turing machine,but they are no less interesting. Theyare called finite-state machines or fi-nite-state automata, and they establishthe minimum specifications of a working digital computer.

Properly defining a finite-state ma chine call s for a degree of mathemat icalrigor that is not appropriate here . Thenature of the concept can be made clear,ho wever, by means of a few examples.When I went out Looking fo r finitestatemachines, I found an excellent specimenin a sta tion of the Lexington A venuesubway in New York. It is a turnstile , anold one made no t with the compact steeltripod of current practice but with fouroak crossarms, worn smooth by a riverof hands and hips.The turnstile has two states: lockedand unlocked. Suppose it is in the lockedstate, so that the arms cannot be turned .Putting a token into the slot a lters theinternal mec hanism in som e way tha tallows the arm s to mo ve; in othe r words,the token induces a transition to the unlocked state. Rotating the arms by 90degrees causes another transition thatrestores the turnstile to the locked state.The transitions are shown schematically in the LIpper illustration on the nextpage. The sta tes of the system are represented by nodes (boxes) and the tran sitions by arcs (arrows) between them.

I n the finite-state anal ys is of the turnst ile, inserting a token and pushing onthe arm s are the possible inputs to thesystem. The response of the machine de pends both on the input and on the stateat the time of the input. Pushing on thecrossarm when the turnstile has not yet

received a token will no t gt t yo u a rideon the subway. Inserting a to ken whenthe ar ms are already unlocked is al sofutile, although in a slightly differentway. The second token is accepted, butit has no effect on the stale of [he machin e; one person is admitted and thenthe turnstile locks again. Three or fourtokens in sequence are likewise accep t-ed but buy only one ride . Skeptics ma ywant further evidence before acceptingthe generalization that all tokens afterthe first have no effect, but they willhave to supply their own tokens .

The reason the turnstile cannot givemultiple rides for multiple tokens is thatit has no means of counting the tokens ithas received. Its onJy form of memory isa rudimentary one: by changing fromone state (0 the other it " remembers"whether the most recent input was a token or a push on the cro ssarms. All earlier inputs are lost. It is worth noting thatthis forgetfulness can never work to thedisadvantage of the city. It could beworse: a turnstile could be designed tochange state after every token, regardless of the present state, in wh ich casetwo tokens in a ro w would admit no one.

The turnstile illustrates most of theesse nt ial properties of a finite-sta temachine . Ob viollsly the ma chine m us[ha ve some sta tes, and there ca n be onlya finile number of them. There canbe inputs and outputs associat ed witha ny state. The states must be discrete ,or clearly distinguishable, and the tran sitions between them must be effective lyinstantaneous. In these matters muchdepends on the point of view: day andnight are discrete states if one is wi llingto define sunrise and sunset as insta ntaneous processes. The set of states, theinputs and the outputs constitute the en-tire machine; there ca n be no a uxiliarydev ices, and in particular no facilitiesfor the storage of information .

The rlil es for building a finite-sta temCichine al low some scope for variation.There arl;: deterministic and nondeter mini stic machines, Moore machines andMealy machines. In a deterministic machine a given input in a given state inv ar

iably has the same result; in a nondeterminist ic machine there can be severalpossible transitions. In the Moore ma chine (named for Edward F. Moore)each state has a unique output. In theMealy machine (named for G . H. Mealy) the outputs are associated with thetransitions ra ther lhan the states. I t turnsout , ho wever, that th e variety of architectures is something of an illusion. Anytask that can be done by one kind offinite-state machine can be done bythe other kinds as well, although thenumber of sta les needed ma y vary.Here I shall discuss mainly deterministic Moore machine s, which have the simplest structure.

W hen yo u start loo king for finitesta te machines. you find Ihem everywheTe. Co in-operated devices are favorite textbook examp les. Some vending ma chines are less rapaciolls thanthe subway turnstile: once they have rcceived the proper amoun t of moneythey enter a state in wh ich all additional coins are rejected. The co in-operateddevice with the largest number of possible states is sure ly the Las Vegas slotmachine. In principle it is deterministic,but finding an input (a coin an d a plJIlon the handle) that will CCi LI se a transition to a partic ular final sta te is nonetheless challenging.

Many ho usehold ap p liances can beregarded as fulite- state machines, although the y tend to be rather dull ones.A clothes washer goes through an inflexible sequence of states- filling, agitating, rinsing, spinn ing- a nd the fewmeaningful inputs, such as plilling theplug out of the electric outlet, generallyhave the same effect in all the states.Similarly, a traffic light has a sma ll repertory of states, which repeat indefinitely. To me the mo st boring of all fini[estate machines is a digital clock. I f itdisplays the month, the date and the passage of hours, minules and seconds, ithas som e 31 million sta tes; in the courseof a yea r it visits each state exactly once.A mousetrap is a finite -state machine;the mouse, 'usually to its misfortune,triggers a transition from the cockedstate to the sprung state. A comb inationlock is a finite-state machine with manypossible inputs, only one of which causes a state transition. A telephone hassta tes that might be labe led on hook, of fhook, waiting, dial tone, dialing. rin ging,connected and out of order. An automo:-bile ca n demonstra te vividl y tha l the ef-fec t of an input varies according to thepresen t state of the sys tem . What happens whe n you press the acceleratorpedal to the floor? I t depends . Is the en gine running? Is the clutch engaged? Isthe parking brake off? Is the transmis sio n in gear? Is it in forward or reverse?Is the garage door open?In the living cell the molecular sys temmade lip of the ribosome and the va ri

19

7/28/2019 SciAm 1983 12 Hayes FSA

2/6

START PUSHLOCKED UNLOCKED

TOKENPUSH TOKEN

A state-transition diagram fo r (1 subway t u r l l . ~ l i l f !

START-- ;> iEVEN ODD

The parity-testing machif/c

Ol i S spec ies of transfe r RN A operatesas a finite-state machine. The inputs arethe fo ur nucleotide bases of messengerRNA, designa ted by the abbreviationsU. A. G and C. The outputs are the 20amino acid components of proteins. i\chain of n ucleotides is recogni zed as avalid inpu t to lhe machine onl y if it begins wi th the "start" signal A UG. Th e reafter the machine reads the input streamcon tinuously, changing sta te as each co don, or tri ple t of nucleotides. is recognized. Th e three special codons UAA,VAG and UGA ar e "stop" slgnals: whenone of th em is en co untered, the ma chinehalls. Many other biological sys temscan use fully be represented as fini testa te machines; examples that come tomind arc the he moglobin molecul e andthe promoter and repressor proteins ofbacteria.In the theology of Th omas Aquinasthe so ul is a finite -s tate machine, a wonderfully e laborate and full y de ter ministic onc. I t is crea ted in a state of jeo pardy, as a conseq uence of original sin. Onbaptism it enters a state of grace, bUlcertain acts (idolat ry, blasphemy, adultery and so forth) induce a transition to astate of sin. ConfeSS ion , repentance andabsolution arc the n needed to res to re theso ul to grace. Th e e ffec t of a fin a l input,dea th , depends critica ll y on the state ofthe so ul at the moment of dea th: in astate of gra ce death leads to salvationbut in a sta te of sin it leads to d a m n ation . The soul ma chine is actually ma rccomplicated than thi s description suggests. A full accoun t wo uld ha ve to distin gu ish among the various grades of sin(venial and mona l, actual and habi tual)and would have to include other possible states of the so ul (such as those associated with limbo and purga tory) andother possible inputs (such as the La stJ udgmenl) .

In quan tum mechanics even the atombecomes a finite-state mac hine , andhencc so docs every thing th at is made upof atoms. The states of the atom arc thea llowed energy le vel s; the inputs a ndou tp uts are photon s, or quanta of eJcctrom agnelic radiation. In a precise description I think the atom wo uld bec lass ified as a nondeterministic Me a lymachine with e psilon tTansitions. I t isnondeterministic beca use the effect ofan input ca nnot be predicted with certainty. I t is a Mealy mach ine beca usethe nature o f the output (namely the en ergy of the photon) is de term ined bythe tra nsition, not by the stat.e entered .Epsilon transitions arc those that cantake place in the absence o f any input;they mu st be included in the model beca use an a to m can em it a phoLOn andchange its state spontaneo llsly.

I s the brain a fini te -state machine? As ithappens, the modern st ud y o f fini testate systems began with a mode l ofneural ne two rk s introduced in 1943 byWarren S. McC ulloch and Walter Pius.T he neurons of McC ulloch and Pittswere simple cells with exc itatory and inhibitory inputs ; each celt had a singleoutput and two inter nal states: firing andnot firi ng. The ce ll s co uld be arrangcdin networks to ca rry out various logicfunct ions, including the "a nd, " "or" and"no t" fun c ti ons that ar e now common place ck ments of electron ic logic systems. The equivalence of the idea lizedneura l network s to sta te-transiti on diagrams of the kind shown here was es tabli shed in 1956 by Stephen C. Kleene ofth e University of Wiscon sin a l M adison.Forty years after the work of McC ul loch and Pitts it is still subject to dispute whether the bra in ca n re asonab lybe classified as a finite-s tate system . Ofcourse the number of neurons is neccs

sarily finite, but that is not the only issue.A rea l ne uron is far more co mplica ted than a two-sta te cell, and so me ofits properties may vary over a cont inuous ra nge rather than being co nstrainedto occ upy discre te states. Furthermore,the prohibition of auxiliary informationstorage in a finite-sta te model of thebrain is aw kward a t best. If menta l life jsno more than a s llccession of instantaneous states, without knowledge of itsown histo ry, then what is memory?

The states of mind discllssed in psy chology , suc h as boredom , fear, thirs t,ecs tasy and grief, seem to fit more read ily into the apparatu s of a noite-state theor y. On the other hand , the sta tes areso numero lls and the transitions arc sopoor ly understo od that the m odel is use

c s s ~ Only for lowe r animals is it possibleto draw more than iso lated fragmentsof the sta te-transition diagra m, and inthose species the experimenter can haveno dir ec t access to the presumed mentalstates. Indeed, mu ch work of this kindha s been done by behaviorists who denythe very existence of mental states.

The case of the digita l comp uterand here ' mean the tangible machine,the hardwa re-is also problematic. Th ecommon me ntal mo del of a computer ,formul ated by John von Ne um a nn, di vides the machine into a cent ral processing un it and an a rray of memory ce lls.There is no doub t that the f 1 n i t t a t eco ncept ca n be applied to the v

7/28/2019 SciAm 1983 12 Hayes FSA

3/6

sary . Whereas the hardware knows onrybinary integers and simple comm andsfor moving and manipula ting them, thevirtual computer dea ls with fa r morcexpress ive symbol systems: words, eq ua tions, arrays, fun ctions. vectors, codons,lists, images. pe rhaps even ideas. Fini testa te techniques can be va lua ble increa ting the virtual computer , and sometimes the virtual computer is a finit e-sta te machine.Consider a program whose ob ject is torea d a series of binary digits (1 's and 0'5)an d report whether the num ber of l '5rece ived is even Or od d. (The task hasprac tical significance; for exampl e, suchparity-check in g program s arc em ployedto detcc t c rrors when d igita l da ta aretransmitted by telephone.) Th e prog ramca n be con structed as a Rnile-sta te machine with two sta tes, as is show n in thelower illustration on the o ppos ite page.Operation begins in the even sta te, because initially no 1 s have been rece ivedand 0 is considered an eve n n urn ber.Each 1 in the input strea m causes achange of state, wh ereas a 0 rece ived ine ither state leaves the state unchanged.Even lhough the mac hine cannot "remember" an y inputs be fore the most recc nt onc and certainly ca nnot count the1 s or 0'5, its output aJways re flee rs theparit y of the input strea m.

The finite-stat e model o f computa tion

START

is commonest in progra ms tha t dea l insome way with tex t or other informa tionthat takes a linguis ti c form. The preeminent exampl e is fo un d in compilers: programs tha t translate programming statements in a so urce lang uage into equivalent statements in a target language,most often the "machine language" ofa particul a r compute r. Compilers andother transla ting prog rams are essentia lto the notion of the virtua l ma chine;the y media te be tween symbo ls with human mea ning and those recogn ized bythe computer .

The part of a co mpiler that can bedesigned as a finite-state machine iscalled the lexica l scanner. Like the su bway turnstile, it is a token-gobbling device. In this case, howe ver, the tok ensare the words , or fundamenta l lex ica lunits, of the language. T he scanner ex amines each gro up of charac ters and determines whe ther it is a genuine to ken,sllch as a com mand or a numbe r; if it isnot , the scanne r rejec ts it as nonsense,just as the lurnstile wo uld reject a slug.The op era tion of a lexical scanner canbe illustra ted by a fmite-state m achi nedesigned to recognize the tokens of asimple language, albeit one of limitedexpressive range: the tokens consist exclusive ly o f Ro man numeral s. Ind eed,only Ro man numeral s of a specia l formare accepted; they must be gi ven in stric t

additive no ta tion, so tha t 9 is represe nt ed by VIIlJ rather .han by IX. (Th ere isevide nce tha t the Romans themselvesemp loyed the additive no ta lion ; the subtrac tive form is thought to have been aGerma n innovation.)A state-transition diagram for the Ro man -numera l machine is shown in theill ustra tion on the next page. Its alph abe t of input symbols includes the le tters M , D, C, L , X, V and I as we ll asthe space symbol , or blank. Any initia l blanks are simpl y ignored, bu t oncethe nrst letter is rece ived the programma kes an immediate transitio n to a s ta teiden ti fie d (for con ven ience) by the nameof th e letter. I f the fi rst lette r is an M, itca n be followed by an y character fro mthe allowed set, including a nother M. I fthe next character is a D, however , thesituat ion is different. From the 0 stateno transition back to the M sta te is defined, because an y series of sym bols thatincludes OM cannot be-a we ll -formedtoken in the language of addi t ive Roman numerals. Furtherm ore, there is notransition from the D sta te to the D sta teilself, so that DD is a lso an excludedseq uence . (The reason is tha t the "hCllfvalue" 'symbols D , L and V ca nno t berepea ted in proper Roman numerals.)In the 0 state the only recognized letters are the lower-valued ones C, L, X, Van d l. The same set is accepted in the C

LASTJUDGMENTLIMBO

DEATH

ORIGINAL SINBAPTI SM

SALVATION

LASTJUDGMENTPURGATORY

EXPIATION

DEATH DE ATHMURDER. ETC.r- ----;.SLOTH. ETC. MURDER. ETC. MORTAL SINGRACE VENIAL SIN

ABSOLUTION MUR DER, ETC.'

7/28/2019 SciAm 1983 12 Hayes FSA

4/6

ISTART-7

BLANK'V

M BLANK0 ,C VL M XX - LV - C, M 0V

rBLANK,

'7 0 VXLC

'"LANK,' - 7 c V

r cXL~,.

~ BLANK,V ~X

BLANKX , f-X V~

) V B L A N ~

,[,I, BLANK1, ,

I

I END IA fexica l !i f.:allller Jo/' a lal1guage of Roman lIumel'ftfs

state (because C ca il be repe ated), but inthe L state onl y the leLters X, V and f :irerecognized. The rule go ve rning the tra nsi tions should be cle ar. T he states arcarranged in a hierarchy, and once a giv en level has been reached the mach inecan never return to a h igher le vel; in thehalf-va lue levels it cannot even remainat the sa me leveL By the time the I stateis reached onl y an ad ditional Jor a blankis a llowed. The blank, entered at thi spoint or a1 any other li me af ter the firs tle tte r. indicates the end of the tokenand sends the machine back to it s sta rting state, read y to rece ive the next Roma n numera l.

N o programming language known tome allows numbers to be entered inRo m an fo rm, but virtuaHy all such languages ha ve faci lities for handling A rabic numbers. The techniques for recogn ition arc similar, a lthough there is agreater va riety of formats. Simp le in tegers such as 137 can be ha ndled inprinciple by a one -state machine. butthe severa l parts of a number such as+6.625 X 10 - 27 req uire a mo re elaborate lex ica l analysis.

Th e ribosome-transfer- RNA systemca n be regarded as a lexical scanner thatrecognizes biologica lly me a ningful nucl eo tide sequences in a.mo lecu le of me s-senger R NA. To be ac cepted a sequencemllst begin with a start co don a nd en dwith one of the three stop co dons; between these bounda r ies any combina tion of the input symbols U, A, G and C,taken th ree at a time, is a llowed.Lex ica l a nalysis is only the firs t step inthe pr ocess of compilation ..The c o m p onents of the comp ile r tha t are called intoac tion after the lexica l scanner are theparser and the code genera to r. The parser takes as its input the tokens id en tifiedby the scanner an d a nalyzes the ir syntactic re lations; this is the closest thecompi ler comes to understanding theme aning of the progr.am statements ittranslates. Th e co de generawr writes aprogram in the target language that ca rrie s ou t th e functions specified by theparsed statements .For the toy languagesconsidtred herethe tasks of the parser and the code gen

erator are tri vial. The compiled fo rm ofa statement in 'the Ro m an -numeral language might be sim ply the Ar abic eq u ivalent of the number. It could be generated by the following strategy. Before a token is scanned a storage ce ll isspecifie d and is se t eq ua l to zero. Theneach time the scanner en ters the M state1,000 is added to the va lue in the ce ll ;for the D state 500 is added, and soon. When the scanning is co m ple te, thememory ce ll holds the va lue of the Rom an numeral. Note tha t the to y co mpiler is no lo nger a pure fin ite-state ma chine, because it ha s aux iliary storage.A com piler for the genetic code iseven simpler and can be rea lized entir e

22

http:///reader/full/progr.amhttp:///reader/full/progr.amhttp:///reader/full/progr.am

7/28/2019 SciAm 1983 12 Hayes FSA

5/6

ly within [he context of a finite -s tate system. The compiled " program " is a sequence of the standard three-Ieuer sy mbols for amin o acids; the symbols can begenerated as the output of the states ofthe scanner thaI recognize codons. Thethree 's tates corresponding to stop codons ha ve no output.Crea ting a comp iler for a languagelarge enough to be of genera l utility isnOl a casual undertaking, but the underlying architecture of the finite-state machine can at least provide an organizingprinciple. I f the syntax of the language isspec ified with suffi cient precision, partof the work can even be mechanize d: itcan be done by a compiler compiler, aprogram whose input is 1 Arg ,~ A G ~S -G ~~~ ~ ~ ~gP- ~?lliB~ ~ p -

U

G

U :Jy: J - - -

r ~ ~U ~ AlaeU I

A ~ ~P )LQI&y ~ ~

A j i f l i l e - ~ / a l e machine (mll.tlttleS the gellel ic code i ll io pro/eill

STOP

27

7/28/2019 SciAm 1983 12 Hayes FSA

6/6

--

rcason that the s ubtrac t ive notation forRoman numerals cannot be handledby a finite-state ma chine. I f the expression X I is read and the machine interpre ts it as It , it cannot go back to revisethe value when the next character turnsout to be V. Many other functions areruled out by the same limitation. Fo rexample, it is no t possible to build afinite-state machine that reads a se"quence of binary digits and determineswhether the number of l' s is equal tothe number of O's. Similarly , a lthougha finite-state machine can add binarynumbers. it ean no r muhiply them; Ileave it 10 the reader 10 deduce why.

Beyond finite-state machines and rcgular languages there extends a hierarchy of more powerful machines andmore general languages. It is called theChomsky hierarchy, after the linguistNoam Chomsky, who investigated thevarious formal languages as poss iblemodels of natural language. The morcgeneral la nguages are created by relaxing constraints on the gra mmatica l rulesof reg ular sets; the machines are built byaddlng memor y clemen ts to the basicfinite -stalc model.

Thc next machine in the series iscalled the pushdown au tomaton. It consists of a finite-state machine with theaddition of a memory array that has aninfinite capacity but a peculiar organization. The memory takes the form of astack, like a counterweighted stack ofcafeteria trays. An item of informationcan be stored only by putting it on top ofthe stack ; when the information is re trieved, any o ver lying items must first beremoved. Thu s the last item in is the firstone out.The language recognized by a pushdown automaton is called a context-freelanguage. In parsing its statements lheacceptability of a symbol can dependboth on the symbol immediately to theleft and on the one immediately to theright. This bidirectional dependency ispermissible because any symbols whoseinterpretalion cannot be decided imme-

PUS HDOWN AUTOMATON

- FINITESTATEMACHINE

, , SEMI-INFINITE: STACK

diately can be stored on the stack until the ambiguity is resolved. Hence apushdown automaton can work withsubtractive Roman numerals, and it canidentify expressions with equa l numbersof 1's and O's (o r other symbols, such asleft and right parentheses). On the other hand, it cannot detect sentences withequal numbers of three symbols, s uch asO's, l' s and 2'5. Most programming languages are context-free, and the parserof a compiler is generally a pushdowna utomaton . Many computers includehardware facilities for organizing a partof the memory capacity as a pushdownstack. One programming language,Forth, makes a stack the primary memory structure. Of course, the stack in anyrea l machine cannot have infinite depth.

The context-free languages merittheir name because the parsing of anysymbol can be influenc-cd directly onlyby the symbol's two immediate neighbors, not by the wider context in which itis found . Removing this constraint givesrise to a context-sensitive language andonce again increases the difficulty ofinterpretation. Now widely sep aratedsymbols can interact; in the worst case itis not possible to interpret the first symbol in an expression until the last onehas been rcad. In exchange for the addedcomplexity somewhat greater capabilityis gained. A machine based on a conlexlsensitive language can determine whether an expression incl ud es equal numbersof three symbols.

The machine that can recognize aco ntext -sensitive language is a linearbounded automaton. In addition to theusual finite -state apparatlls it has amemory organized in such a way thatany storage location can be reached atany time; it is a random-access machine.The memory is only finite in capacity,bu t it is assumed to be large enough [ 0hold any input the machine rece ives.The linear-bounded a utomaton see ms agood approxima tion of lhe von . Neu mann model of a digital computer. Odd ly, thollgh , the corresponding con textsensitive programming languages seem

LINEAR-BOUNDED AUTOMATON

FINITESTATEMACHINE

FINITEMEMORY ARRAY

to be rare; evidently the simpler contextfree structure almost always has .sufficient.expressive power.All the languages described abovehave a property in common: they arcsaid to be recursive. What this designation amounts to is that one can imag inea procedure for genera tin g all possible" utterances" in the language in order ofincreasing length. It follows that there isa guaranteed method of deciding whet.her any given statemen t of finite length 'jsa member of the, language:, simply gen erate all the statemcnts,up to that lengthand compare t!x.m.There are.languages that cannot meeteven this minimal standard of tractab ility. For them Ihere is only one poss iblerecognizing machine: it is the computerof last resort, the Turing mac hine, a fi nite-stale automaton a llowed to roamfreel y through an unbounded memory.In the description given by Turing thememory is a tape, infinite in both directions and marked off into cells, whichthe finite -s tate apparat us can write on,read or erase.

Looking down rrom the elevated perspective of the Turing machine. the relations of the lesser comput ing devicesbecome clearer. Th e hnear-boundedautomaton is s imply a Turing machinewith a finite tape. The pushdown all tom aton has a tape that is infinite in onedirection, bu t the "head" for readingand writ ing On the [;;Ipe always remainsfixed over the last non blank cell. Th etinite-state machine is a T uring machinewith no tape at alLBrand -name -conscious readers, eagerto parse nonrecllfsive languages, mayalready be out shopping for a Tur ingmachine. They should be warned thatthe ultimate compu ter a lso ha s irs weaknesses. There are languages wit h grammilrs so preposterous (hat even il Turingmac hine ca nnot be counted on to recogni ze the ir statements in a finite amountof time. So far such l;;lnguages havefound little use in the world of computing machines, bu( people somehowmanage to speak them .

TURING MACHINE

.

FINITESTATEMACHINE

, , INFINITEi MEMORY ARRAYl1w Chomsky hieranhy of finiU! (/11d infinite mochines

28