theory of computation - 北海道大学thomas/toc/slides/toc_lec03.pdf · introduction finite...

Introduction Finite Automata Automata and Regular Languages Applications End

Theory of Computation

Thomas Zeugmann

Hokkaido UniversityLaboratory for Algorithmics

http://www-alg.ist.hokudai.ac.jp/∼thomas/ToC/

Lecture 3: Finite State Automata

Theory of Computation c©Thomas Zeugmann

Motivation

In the previous lecture we learned how to formalize thegeneration of languages, i.e., we looked at formal languagesfrom the perspective of a speaker.

Now, we turn our attention to accepting languages, i.e., we aregoing to formalize the perspective of a listener.

In this lecture we deal with regular languages, and the machinemodel accepting them.

Motivation

Overall Goal

Let Σ be again an alphabet, and let L ⊆ Σ∗ be any regularlanguage. Now, for every string s ∈ Σ∗ we want to have apossibility to decide whether or not s ∈ L.

Looking at the definition of a regular grammar, the followingmethod may be easily discovered. We start generating stringsuntil one of the following two conditions happens:(1) The string s is generated. Clearly, then s ∈ L.(2) The length of our string s is exceeded. Taking into account

that all further generable strings must be longer, we mayconclude that s < L.

But this method lacks efficiency. It may take time that isexponential in the length of the input string s to terminate.

Overall Goal

Nondeterministic Finite Automata I

The approach described at the previous slide hardly reflectswhat humans are doing when accepting sentences of naturallanguages. Therefore, we favor another approach.

Idea: Looking at a regular grammar, we have productions

σ → a

σ → aσ

Now, we simply change the direction of the → , i.e.,

σ ← a

σ ← aσ

σ → a

σ → aσ

σ ← a

σ ← aσ

σ → a

σ → aσ

σ ← a

σ ← aσ

Nondeterministic Finite Automata II

This leads to the following definition:

Definition 1 (NDFA)

A 5-tuple A = [Σ, Q, δ, q0, F] is said to be a nondeterministicfinite automaton if(1) Σ is an alphabet (the so-called input alphabet);(2) Q is a finite nonempty set (the set of states);(3) δ : Q× Σ→ ℘(Q), the transition relation;(4) q0 ∈ Q, the initial state, and(5) F ⊆ Q, the set of final states.

In the definition above, ℘(Q) denotes the power set of Q, i.e.,the set of all subsets of Q.

Nondeterministic Finite Automata II

This leads to the following definition:

Definition 1 (NDFA)

A 5-tuple A = [Σ, Q, δ, q0, F] is said to be a nondeterministicfinite automaton if(1) Σ is an alphabet (the so-called input alphabet);(2) Q is a finite nonempty set (the set of states);(3) δ : Q× Σ→ ℘(Q), the transition relation;(4) q0 ∈ Q, the initial state, and(5) F ⊆ Q, the set of final states.

In the definition above, ℘(Q) denotes the power set of Q, i.e.,the set of all subsets of Q.

Deterministic Finite Automata

There is also a deterministic counterpart of a nondeterministicfinite automaton which we define next.

Definition 2 (DFA)

A 5-tuple A = [Σ, Q, δ, q0, F] is said to be a deterministic finiteautomaton if(1) Σ is an alphabet (the so-called input alphabet);(2) Q is a finite nonempty set (the set of states);(3) δ : Q× Σ→ Q, the transition function, which must be

defined for every input;(4) q0 ∈ Q, the initial state, and(5) F ⊆ Q, the set of final states.

Deterministic Finite Automata

There is also a deterministic counterpart of a nondeterministicfinite automaton which we define next.

Definition 2 (DFA)

A 5-tuple A = [Σ, Q, δ, q0, F] is said to be a deterministic finiteautomaton if(1) Σ is an alphabet (the so-called input alphabet);(2) Q is a finite nonempty set (the set of states);(3) δ : Q× Σ→ Q, the transition function, which must be

defined for every input;(4) q0 ∈ Q, the initial state, and(5) F ⊆ Q, the set of final states.

Computing with Finite Automata I

When we do not want to specify whether an automaton isdeterministic or nondeterministic, we simply refer to it as afinite automaton.

So far, we have explained what an automaton is but not what itdoes. In order to explain how to compute with an automaton,we need some more definitions. For the deterministic case, wecan easily define the language accepted by a finite automaton.

First, we extend the definition of δ to strings. That is, formallywe inductively define a function

δ∗ : Q× Σ∗ → Q ,by setting

δ∗(q, λ) = q for all q ∈ Q ,δ∗(q, sa) = δ(δ∗(q, s), a) for all s ∈ Σ∗, a ∈ Σ, and q ∈ Q .

Computing with Finite Automata II

Furthermore, we also need the following lemma:

Lemma 1Let A = [Σ, Q, δ, q0, F] be a deterministic finite automaton. Then forall strings v, w ∈ Σ∗ and all q ∈ Q we haveδ∗(q, vw) = δ∗(δ∗(q, v), w).

Now, we can define the language accepted by a deterministicfinite automaton.

Definition 3Let A = [Σ, Q, δ, q0, F] be a deterministic finite automaton. Thelanguage L(A) accepted by A is

L(A) = {s | s ∈ Σ∗, δ∗(q0, s) ∈ F} .

Computing with Finite Automata II

Furthermore, we also need the following lemma:

Lemma 1Let A = [Σ, Q, δ, q0, F] be a deterministic finite automaton. Then forall strings v, w ∈ Σ∗ and all q ∈ Q we haveδ∗(q, vw) = δ∗(δ∗(q, v), w).

Now, we can define the language accepted by a deterministicfinite automaton.

Definition 3Let A = [Σ, Q, δ, q0, F] be a deterministic finite automaton. Thelanguage L(A) accepted by A is

L(A) = {s | s ∈ Σ∗, δ∗(q0, s) ∈ F} .

Computing with Finite Automata III

If we have s ∈ L(A) for a string s ∈ Σ∗ then we say that there isan accepting computation for s. We adopt this notion also tonondeterministic automata. Note that λ ∈ L(A) iff q0 ∈ F.

In order to keep notation simple, in the following we shallidentify δ∗ with δ. It should be clear from context what ismeant.

Well, this seems very abstract, and so some explanation is inorder.

Computing with Finite Automata IV

Conceptually, a finite automaton possesses an input tape that isdivided into cells. Each cell can store a symbol from Σ or it maybe empty. Additionally, a finite automaton has a head to readwhat is stored in the cells. Initially, a string s = s1s2 · · · sk iswritten on the tape and the head is positioned on the leftmostsymbol of the input, i.e., on s1.

finite statecontrol

head moves in this direction, one cell at a time

s2s1 sk sk+1

Computing with Finite Automata IV

Conceptually, a finite automaton possesses an input tape that isdivided into cells. Each cell can store a symbol from Σ or it maybe empty. Additionally, a finite automaton has a head to readwhat is stored in the cells. Initially, a string s = s1s2 · · · sk iswritten on the tape and the head is positioned on the leftmostsymbol of the input, i.e., on s1.

finite statecontrol

s2s1 sk sk+1

Computing with Finite Automata V

finite statecontrol

s2s1 sk sk+1

The automaton is put into its initial state q0. Now, it reads s1.Then, it changes its state to one of the possible states inδ(q0, s1), say q, and the head moves right to the next cell. In thedeterministic case, the state δ(q0, s1) is uniquely defined. Next,s2 is read, and the automaton changes its state to one of thepossible states in δ(q, s2). This process is iterated until theautomaton reaches the first cell which is empty. After havingread the whole string, the automaton is in some state, say r. Ifr ∈ F, then the computation has been an accepting one,otherwise, the string s is rejected.

Computing with Finite Automata VI

Now, we see what the problem is in defining the languageaccepted by a nondeterministic finite automaton. On input astring s, there are many possible computations. Some of thesecomputations may finish in an accepting state and some maynot. We therefore define the language accepted by anondeterministic finite automaton as follows:

Definition 4Let A = [Σ, Q, δ, q0, F] be a nondeterministic finite automaton.The language L(A) accepted by A is the set of all strings s ∈ Σ∗

such that there exists an accepting computation for s.

Computing with Finite Automata VI

Now, we see what the problem is in defining the languageaccepted by a nondeterministic finite automaton. On input astring s, there are many possible computations. Some of thesecomputations may finish in an accepting state and some maynot. We therefore define the language accepted by anondeterministic finite automaton as follows:

Definition 4Let A = [Σ, Q, δ, q0, F] be a nondeterministic finite automaton.The language L(A) accepted by A is the set of all strings s ∈ Σ∗

such that there exists an accepting computation for s.

State Diagrams I

Finite automata may be conveniently represented by their statediagram. The state diagram is a directed graph whose nodes arelabeled by the states of the automaton. The edges are labeledby symbols from Σ. Let p and q be nodes. Then, there is adirected edge from p to q if and only if there exists an a ∈ Σ

such that q = δ(p, a) (deterministic case) or q ∈ δ(p, a)

(nondeterministic case). Final states have an extra circle.The state diagram of a finite automaton accepting the languageL = {aibj | i > 0, j > 0} is shown below.

a b a, b

State Diagrams I

Finite automata may be conveniently represented by their statediagram. The state diagram is a directed graph whose nodes arelabeled by the states of the automaton. The edges are labeledby symbols from Σ. Let p and q be nodes. Then, there is adirected edge from p to q if and only if there exists an a ∈ Σ

such that q = δ(p, a) (deterministic case) or q ∈ δ(p, a)

(nondeterministic case). Final states have an extra circle.The state diagram of a finite automaton accepting the languageL = {aibj | i > 0, j > 0} is shown below.

a b a, b

Illustrating the Example

First, let us consider the computation performed by the statediagram on input abb.

Having read a

31 2b a

a, bba

Having read ab

31 2b a

a, bba

Having read abb

31 2b a

a, bba

Thus, the string abb is accepted.

Second, let us consider the computation performed by the statediagram on input ba.

Having read b

31 2b a

a, bba

Second, let us consider the computation performed by the statediagram on input ba.

Having read ba

31 2b a

a, bba

Thus, the string ba is rejected.

Automata and Regular Languages

Now that we know what finite automata are, we can answerthe question you probably have already in mind, i.e.,

QuestionWhat have finite automata to do with regular languages?

For answering this question, we show the following theorem:

Automata and Regular Languages

Now that we know what finite automata are, we can answerthe question you probably have already in mind, i.e.,

QuestionWhat have finite automata to do with regular languages?

For answering this question, we show the following theorem:

Main Theorem

Theorem 2

Let L ⊆ Σ∗ be any language. Then, the following three assertions areequivalent:(1) There exists a deterministic finite automaton A such that

L = L(A).(2) There exists a nondeterministic finite automaton A such that

L = L(A).(3) L is regular.

We show the equivalence by proving (1) implies (2),(2) implies (3), and (3) implies (1).

Proof of the Main Theorem

Claim 1. (1) implies (2).

This is obvious by definition, since a deterministic finiteautomaton is a special case of a nondeterministic one.

Let A = [Σ, Q, δ, q0, F] be a nondeterministic finite automatonsuch that L = L(A). We have to construct a grammar G

generating L. Let G = [Σ, Q ∪ {σ}, σ, P], where P is thefollowing set of productions:

P = {σ → q0} ∪ {p → aq | a ∈ Σ, p, q ∈ Q, q ∈ δ(p, a)}

∪ {p → λ | p ∈ F} .

Obviously, G is regular. We have to show L(G) = L(A). First weprove L(A) ⊆ L(G).

Let s = a1 · · ·ak ∈ L. Then, there exists an acceptingcomputation of A for s.Let q0, p1, . . . , pk be the sequence of states through which A

goes while performing this accepting computation. Therefore,p1 ∈ δ(q0, a1), p2 ∈ δ(p1, a2), . . ., pk ∈ δ(pk−1, ak), and pk ∈ F.Thus, σ ⇒ q0 ⇒ a1p1 ⇒ · · · ⇒ a1 · · ·ak−1pk−1 ⇒a1 · · ·akpk ⇒ a1 · · ·ak. Hence, s ∈ L(G).

The direction L(G) ⊆ L(A) can be proved analogously, and istherefore left as an exercise.

Let s = a1 · · ·ak ∈ L. Then, there exists an acceptingcomputation of A for s.Let q0, p1, . . . , pk be the sequence of states through which A

goes while performing this accepting computation. Therefore,p1 ∈ δ(q0, a1), p2 ∈ δ(p1, a2), . . ., pk ∈ δ(pk−1, ak), and pk ∈ F.Thus, σ ⇒ q0 ⇒ a1p1 ⇒ · · · ⇒ a1 · · ·ak−1pk−1 ⇒a1 · · ·akpk ⇒ a1 · · ·ak. Hence, s ∈ L(G).

The direction L(G) ⊆ L(A) can be proved analogously, and istherefore left as an exercise.

In principle, we want to use an idea similar to the one used toprove Claim 2. But productions may have strings on their righthand side, while the transition function has to be defined overstates and letters. Therefore, we first have to show a normal formlemma for regular grammars.

Normal Form LemmaFor every regular grammar G = [T , N, σ, P] there exists agrammar G ′ such that L(G) = L(G ′) and all productions of G ′

have the form h → ah ′ or h → λ, where h, h ′ ∈ N and a ∈ T .

In principle, we want to use an idea similar to the one used toprove Claim 2. But productions may have strings on their righthand side, while the transition function has to be defined overstates and letters. Therefore, we first have to show a normal formlemma for regular grammars.

Normal Form LemmaFor every regular grammar G = [T , N, σ, P] there exists agrammar G ′ such that L(G) = L(G ′) and all productions of G ′

have the form h → ah ′ or h → λ, where h, h ′ ∈ N and a ∈ T .

Proof of the Normal Form Lemma

First, each production of the form h → a1 · · ·akh ′, k > 0, isequivalently replaced by the following productions:

h → a1ha2···akh ′ , ha2···akh ′ → a2ha3···akh ′ , . . . ,hakh ′ → akh ′ .

Next, each production of the form h → a1 · · ·ak, k > 0, isequivalently replaced by the following productions:

h → a1ha2···ak, ha2···ak

→ a2ha3···ak, . . . ,

hak→ akhλ and hλ → λ .

Finally, we have to deal with productions of the form h → h ′

where h, h ′ ∈ N.

→ a2ha3···ak, . . . ,

where h, h ′ ∈ N.

→ a2ha3···ak, . . . ,

Proof of the Normal Form Lemma continued

Let G = [T , N, σ, P] be the grammar constructed so far.Furthermore, let U(h) =df {h ′ | h ′ ∈ N and h

∗⇒ h ′}.

Clearly, U(h) is computable. Now, we delete all productions ofthe form h → h ′ and add the following productions to P:If h ′ → λ ∈ P and h ′ ∈ U(h), then we add h → λ. Moreover,we add all h → xh ′′ for all productions in P such that there is ah ′ ∈ U(h) with h ′ → xh ′′ ∈ P.

Let G ′ be the resulting grammar. Clearly, now all productionshave the desired form. One easily verifies that L(G) = L(G ′).This proves the lemma.

∗⇒ h ′}.

Illustration of Normal Form Lemma

Let G = [{a, b}, {σ, h}, σ, P] be the grammar given, whereP = {σ → abσ, σ → h, h → aah, h → λ}.Applying the transformation, first we obtain:P = {σ → ahbσ, hbσ → bσ, σ → h, h → ahah, hah →ah, h → λ}.Now, U(σ) = {h}, and U(h) = ∅. Thus, we delete the productionσ → h and replace it by σ → ahah and σ → λ.Summarizing this construction, we now have the following setP ′ of productions:

P ′ = {σ → ahbσ, hbσ → bσ, σ → ahah, σ → λ,h → ahah, hah → ah, h → λ}

as well as the following set N ′ of nonterminals:

N ′ = {σ, hah, hbσ, h} .

Proof of (3) implies (1) continued

Now, assume that we are given a grammar G = [T , N, σ, P] thatis already in the normal form described in the lemma above.We define a deterministic finite automaton A = [T , Q, δ, q0, F]

as follows:

Let Q = ℘(N) and q0 = {σ}. The transition function δ is definedas

δ(p, a) = {h ′ | ∃h[h ∈ p and h → ah ′ ∈ P]} .

Finally, we set

F = {p | ∃h[h ∈ p and h → λ ∈ P]} .

We leave it as an exercise to prove L(A) = L(G).

as follows:

δ(p, a) = {h ′ | ∃h[h ∈ p and h → ah ′ ∈ P]} .

Finally, we set

F = {p | ∃h[h ∈ p and h → λ ∈ P]} .

as follows:

δ(p, a) = {h ′ | ∃h[h ∈ p and h → ah ′ ∈ P]} .

Finally, we set

F = {p | ∃h[h ∈ p and h → λ ∈ P]} .

Deciding Membership

Theorem 2 directly allows for the following corollary:

Corollary 3There is an algorithm that on input any regular grammarG = [T , N, σ, P] and any string s ∈ T∗ decides whether ornot s ∈ L(G).

Proof. As the proof of Theorem 2 shows, given any regulargrammar G we can construct a deterministic finite automaton A

such that L(G) = L(A). Since A is deterministic, on input anystring s ∈ T∗ it either accepts s or it rejects it.

Deciding Membership

Theorem 2 directly allows for the following corollary:

Corollary 3There is an algorithm that on input any regular grammarG = [T , N, σ, P] and any string s ∈ T∗ decides whether ornot s ∈ L(G).

Proof. As the proof of Theorem 2 shows, given any regulargrammar G we can construct a deterministic finite automaton A

such that L(G) = L(A). Since A is deterministic, on input anystring s ∈ T∗ it either accepts s or it rejects it.

Showing Non-Regularity of Languages I

Finally, we show how to use finite automata to prove thatparticular languages are not regular. Consider the followinggrammar:

G = [{a, b}, {σ}, σ, {σ → aσb, σ → λ}] .

Then, L(G) = {aibi | i > 0}.

Clearly, G is not regular, but this does not prove that there is noregular grammar at all that can generate this language.

Theorem 4

The language L = {aibi | i > 0} is not regular.

G = [{a, b}, {σ}, σ, {σ → aσb, σ → λ}] .

Then, L(G) = {aibi | i > 0}.

Theorem 4

G = [{a, b}, {σ}, σ, {σ → aσb, σ → λ}] .

Then, L(G) = {aibi | i > 0}.

Theorem 4

Showing Non-Regularity of Languages II

Proof. Suppose the converse. Then, by our Main Theorem theremust be a deterministic finite automaton A = [Σ, Q, δ, q0, F] suchthat L(A) = L. Clearly, {a, b} ⊆ Σ, and thus δ(q0, ai) must bedefined for all i. However, there are only finitely many states,but infinitely many i, and hence there must exist i, j such that

i , j but δ(q0, ai) = δ(q0, aj) .

Since the automaton is deterministic, δ(q0, ai) = δ(q0, aj)

implies δ(q0, aibi) = δ(q0, ajbi). Let q = δ(q0, aibi); then, ifq ∈ F we directly obtain ajbi ∈ L, a contradiction, since i , j.But if q < F, then aibi is rejected. This is again a contradiction,since aibi ∈ L. Thus, L is not regular.

Showing Non-Regularity of Languages II

Proof. Suppose the converse. Then, by our Main Theorem theremust be a deterministic finite automaton A = [Σ, Q, δ, q0, F] suchthat L(A) = L. Clearly, {a, b} ⊆ Σ, and thus δ(q0, ai) must bedefined for all i. However, there are only finitely many states,but infinitely many i, and hence there must exist i, j such that

i , j but δ(q0, ai) = δ(q0, aj) .

Since the automaton is deterministic, δ(q0, ai) = δ(q0, aj)

implies δ(q0, aibi) = δ(q0, ajbi). Let q = δ(q0, aibi); then, ifq ∈ F we directly obtain ajbi ∈ L, a contradiction, since i , j.But if q < F, then aibi is rejected. This is again a contradiction,since aibi ∈ L. Thus, L is not regular.

Thank you!

theory of computation - 北海道大学thomas/toc/slides/toc_lec03.pdf · introduction finite...

Documents