Introduction to the theory of computation

Introduction to the Theory of Computation, Michael Sipser
Chapter 0: Introduction

Automata, Computability and Complexity:
• They are linked by the question:
o “What are the fundamental capabilities and limitations of computers?”
• The theories of computability and complexity are closely related. In complexity theory, the objective
is to classify problems as easy ones and hard ones, whereas in computability theory he classification of
problems is by those that are solvable and those that are not. Computability theory introduces several
of the concepts used in complexity theory.
• Automata theory deals with the definitions and properties of mathematical models of computation.
• One model, called the finite automaton, is used in text processing, compilers, and hardware design.
Another model, called the context – free grammar, is used in programming languages and artificial
intelligence.

Strings and Languages:
• The string of the length zero is called the empty string and is written as ε.
• A language is a set of strings.

Definitions, Theorems and Proofs:
• Definitions describe the objects and notions that we use.
• A proof is a convincing logical argument that a statement is true.
• A theorem is a mathematical statement proved true.
• Occasionally we prove statements that are interesting only because they assist in the proof of another,
more significant statement. Such statements are called lemmas.
• Occasionally a theorem or its proof may allow us to conclude easily that other, related statements are
true. These statements are called corollaries of the theorem.

Chapter 1: Regular Languages
Introduction:
• An idealized computer is called a “computational model” which allows us to set up a manageable
mathematical theory of it directly.
• As with any model in science, a computational model may be accurate in some ways but perhaps not
in others.
• The simplest model is called “finite state machine” or “finite automaton”.

Finite Automata:
• Finite Automata are good models for computers with an extremely limited amount of memory, like for
example an automatic door, elevator or digital watches.
• Finite automata and their probabilistic counterpart “Markov chains” are useful tools when we are
attempting to recognize patterns in data. These devices are used in speech processing and in optical
character recognition. Markov chains have even been used to model and predict price changes in
financial markets.
• State diagrams are described on p.34.
• The output of an finite automaton is “accepted” if the automaton is now in an accept state (double
circle) and reject if it is not.

1

PDF created with FinePrint pdfFactory trial version http://www.pdffactory.com

• A finite automaton is a list of five objects:
o Set of states
o Input alphabet
o Rules for moving
o Start state
o Accepts states
• δ ( x,1) = y , means that a transition from x to y exists when the machine reads a 1.
• Definition: A finite automaton is a 5 – tuple (Q, Σ, δ , q 0 , F ) , where
1. Q is a finite set called the states.
2. Σ is a finite set called the alphabet.
3. δ : Q × Σ → Q is the transition function
4. q 0 ∈ Q is the start state
5. F ⊆ Q is the set of accept states.
• If A is the set of all strings that machine M accepts, we say that A is the language of machine M and
write L(M) = A. We say M recognizes A.
• A language is called a “regular language” if some finite automaton recognizes it.
• A finite automaton has only a finite number of states, which means a finite memory.
• Fortunately, for many languages (although they are infinite) you don’t need to remember the entire
input (which is not possible for a finite automaton). You only need to remember certain crucial
information.

The Regular Operations:
• We define 3 operations on languages, called the regular operations, and use them to study properties of
the regular languages.
• Definition: Let A and B be languages. We define the regular operations union, concatenation and
star as follows:
o Union: A ∪ B = {x | x ∈ A or x ∈ B}
o Concatenation: A o B = {xy | x ∈ A and y ∈ B}
o Star: A* = {x1 x 2 ...x k | k ≥ 0 and each xi ∈ A}
• Example: Let the alphabet Σ be the standard 26 letters {a, b, …, z}. If language A = {good, bad} and
language B = {boy, girl}, then:
o A ∪ B = {good, bad, boy, girl}
o A o B = {goodboy, goodgirl, badboy, badgirl}
o A* = {ε, good, bad, goodgood, goodbad, badgood, badbad, goodgoodgood, goodgoodbas, …}
• The class of regular languages is closed under the union operation. In other word, if A and B are
regular languages, so is A ∪ B .
• The class of regular languages is closed under the concatenation operation.
• The class of regular languages is closed under the intersection operation.
• The class of regular languages is closed under the star operation.

Nondeterminism:
• Nondeterminism is a generalization of determinism, so every deterministic finite automaton is
automatically a nondeterministic finite automaton.
• In a DFA (deterministic finite automaton), every state always has exactly one exiting transition arrow
for each symbol in the alphabet. In an NFA (nondeterministic finite automaton) a state may have zero,
one or many exiting arrows for each alphabet symbol.

2


• How does an NFA compute? Suppose that we are running an NFA on an input string and come to a
state with multiple ways to proceed. Fro example, say that we are in state q1 in NFA N1 and that the
next input symbol is a 1. After reading that symbol, the machine splits into multiple copies of itself
and follows all the possibilities in parallel. Each copy of the machine takes one of the possible ways to
proceed and continues as before. If there are subsequent choices, the machine splits again. If the next
input symbol doesn’t appear on any of the arrows exiting the state occupied by a copy of the machine,
that copy of the machine dies, along with the branch of the computation associated with it. Finally, if
any one of these copies of the machine is in an accepts state ate the end of the input, the NFA accepts
the input string. If a state with an ε symbol on an exiting arrow is encountered, something similar
happens. Without reading any input, the machine splits into multiple copies, one following each of the
exiting ε - labelled arrows and one staying at the current state. Then the machine proceeds
nondeterministically as before.
• Nondeterministic finite automata are useful in several respects. As we will show, every NFA can be
converted into an equivalent DFA, and constructing NFAs is sometimes easier than directly
constructing DFAs. An NFA may be much smaller than its deterministic counterpart, or its functioning
may be easier to understand.

Nondeterministic Finite Automaton:
• Definition: A nondeterministic finite automaton is a 5 – tuple (Q, Σ, δ , q 0 , F ) , where
1. Q is a finite set of states.
2. Σ is a finite alphabet.
3. δ : Q × Σ ε → P (Q ) is the transition function, Σ ε = Σ ∪ {ε }
4. q 0 ∈ Q is the start state.
• In a DFA the transition function takes a state and an input symbol and produces the next state. In a
NFA the transition function takes a state and an input symbol or the empty string and produces the set
of possible next states.
• For any set Q we write P(Q) to be the collection of all subsets of Q (Power ser of Q).
• Deterministic and nondeterministic finite automaton recognize the same class of languages.
• Two machines are equivalent if they recognize the same language.
• Every NFA has an equivalent DFA.
• If k is the number of states of the NFA, it has 2 k subsets of states. Each subset corresponds to one of
the possibilities that the DFA must remember, so the DFA simulating the NFA will have 2 k states.
• NFA transforming to DFA:
o The DFA M accepts (means it is in an accept state) if one of the possible states that the NFA N
could be in at this point, is an accept state.
o A language is regular if and only if some NFA recognizes it.

Regular Expressions:
• Definition: Say that R is a regular expression if R is:
1. a for some a in the alphabet Σ .
2. ε.
3. ∅, 1*∅ = ∅, ∅* = {ε}, R o ∅ = ∅
4. ( R1 ∪ R 2) , where R1 and R2 are regular expressions.
5. ( R1 o R 2) , where R1 and R2 are regular expressions.
6. (R1*) , where R1 is a regular expression.
• The value of a regular expression is a language.
• Regular expressions have an important role in computer science applications. In applications involving
text, user may want to search for strings that satisfy certain patterns . Regular expressions provide a
powerful method for describing such patterns.

3


• We can write Σ as shorthand for the regular expression (0 ∪ 1) . More generally, if Σ is any alphabet,
the regular expression Σ describes the language consisting of all strings of length 1 over that alphabet,
and Σ * describes the language consisting of all strings over that alphabet. Similarly Σ * 1 is the
language that contains all strings that end in a 1. The language (0Σ*) ∪ (Σ * 1) consists of all strings
that either start with a 0 or end with a 1.
• Precedence in regular expressions: * > o > ∪
• When we want to make clear a distinction between a regular expression R and the language that it
describes, we write L(R) to be the language of R.
• A language is regular if and only if some regular expression describes it.

Generalized Nondeterministic Finite Automaton:
• Definition: A generalized nondeterministic finite automaton, (Q, Σ, δ , q start , q accept ) is a 5 – tuple where
1. Q is the finite set of states.
2. Σ is the input alphabet.
3. δ : (Q − {q accept }) × (Q − {q start }) → R is the transition function.
4. q start is the start state.
5. q accept is the accept state.
• The GNFA reads blocks of symbols form the input, not necessarily just one symbol at a time as in an
ordinary NFA.
• For convenience we require that GNFAs always have a special form that meets the following
conditions:
o The start state has transition arrows going to every other state but no arrows coming in from
any other state.
o There is only a single accept state, and it has arrows coming in from every other state but no
arrows going to any other state. Furthermore, the accept state is not the same as the start state.
o Except for the start and accept states, one arrow goes from every state to every other state and
also from each state to itself.
• We can easily convert a DFA into a GNFA in the special form. We simply add a new start state with
an ε arrow to the old start state and a new accept state with ε arrows form the old accept states. If any
arrows have multiple labels (or if there are multiple arrows going between the same two states in the
same direction), we replace each with a single arrow whose label is the union of the previous labels.
Finally, we add arrows labeled ∅ between states that had no arrows. This last step won’t change the
language recognized because a transition labeled with ∅ can never be used.
• We let M be the DFA for language A. The we convert M to a GNFA G by adding a new start state and
a new accept state and additional transition arrows as necessary. We use the procedure CONVERT(G),
which takes a GNFA and returns an equivalent regular expression.
• CONVERT(G): Generates a regular expression R out of a GNFA G on p. 73.

Nonregular Languages:
• To understand the power of finite automata you must also understand their limitations. In this section
we show how to prove that certain languages cannot be recognized by any finite automaton.
• Let’s take the language B = {0 n1n | n ≥ 0} . If we attempt to find a DFA that recognizes B, we discover
that the machine seems to need to remember how many 0s have been seen so far as it reads the input.
Because the number of 0s isn’t limited, the machine will have to keep track of an unlimited number of
possibilities. But it cannot do so with any finite number of states.
• Our technique for proving nonregularity stems from a theorem about regular languages, traditionally
called the pumping lemma. This theorem states that all regular languages have a special property. If we
can show that a language does not have this property, we are guaranteed that it is not regular. The
property states that all strings in the language can be “pumped” if they are at least as long as a certain
special value, called the pumping length. That means each such string contains a section that can be
repeated any number of times with the resulting string remaining in the language.

4


• Pumping Lemma: If A is a regular language, then there is a number p (the pumping length) where, if s
is any string in A of length at least p, then s may be divided into three pieces, s = xyz, satisfying the
following conditions:
1. for each i ≥ 0 , xy i z ∈ A
2. | y | > 0
3. | xy | ≤ p
• To use the pumping lemma to prove that a language B is not regular, first assume that B is regular in
order to obtain a contradiction. The use the pumping lemma to guarantee the existence of a pumping
length p such that all strings of length p or greater in B can be pumped. Next, find a string s in B that
has length p or greater but that cannot be pumped. Finally, demonstrate that s cannot be pumped by
considering all ways of dividing s into x, y and z (taking condition 3 of the pumping lemma into
account if convenient) and, for each such division, finding a value i where xy i z ∉ B . This final step
often involves grouping the various ways of dividing s into several cases and analysing them
individually. The existence of s contradicts the pumping lemma if B were regular. Hence B cannot be
regular.
Finding s sometimes takes a bit of creative thinking. You may need to hunt through several candidates
for s before you discover one that works. Try members of B that seem to exhibit the “essence” of B’s
nonregularity.

Chapter 2: Context – Free Languages
Introduction:
• In this chapter we introduce context – free grammars, a more powerful method, than finite automata
and regular expressions, of describing languages. Such grammars can describe certain features that
have a recursive structure which makes them useful in a variety of applications (study of human
languages, compilation of programming languages).

Context – Free Grammars:
• A grammar consists of a collection of substitution rules, also called productions. Each rule appears as
a line in the grammar and comprises a symbol and a string, separated by an arrow. The symbol is
called a variable. The string consists of variables and other symbols called terminals.
• You use a grammar to describe a language by generating each string of that language in the following
manner.
1. Write down the start variable . It is the variable on the left – hand side of the top rule, unless
specified otherwise.
2. Find a variable that is written down and a rule that starts with that variable. Replace the written
down variable with the right – hand side of that rule.
3. Repeat step 2 until no variables remain.
• All strings generated in this way constitute the language of the grammar. We write L(G) for the
language of grammar G.
• Any language that can be generated by some context – free grammar is called a context – free
language (CFL).
• Definition: A context – free grammar is a 4 – tuple (V , Σ, R, S ) , where
1. V is a finite set called the variables.
2. Σ is a finite set, disjoint from V, called terminals.
3. R is a finite set of rules, with each rule being a variable and a string of variables and terminals.
4. S ∈ V is the start variable.
∗
• We write u ⇒ v if u = v or if a sequence u1 , u 2 ,..., u k exists for k ≥ 0 and
u ⇒ u1 ⇒ u 2 ⇒ ... ⇒ u k ⇒ v
∗
• The language of the grammar is {w ∈ Σ* | S ⇒ w}

5


Designing Context – Free Grammars (p.96):
• You can convert any DFA into an equivalent CFG as follows. Make a variable Ri for each state q i of
the DFA. Add the rule Ri → aR j to the CFG if δ ( q i , a ) = q j is a transition in the DFA. Add the rule
Ri → ε if q i is an accept state of the DFA. Make R0 the start variable of the grammar, where q 0 is
the start state of the machine. Verify on your own that the resulting CFG generates the same language
that the DFA recognizes.
• If a grammar generates the same string in several different ways, we say that the string is derived
ambiguously in that grammar. If a grammar generates some string ambiguously we say that the
grammar is ambiguous.
• A derivation of a string w in a grammar G is leftmost derivation of at every step the leftmost
remaining variable is the one replaced.
• A string w is derived ambiguously in context – free grammar G if it has two or more different leftmost
derivations. Grammar G is ambiguous if it generates some string ambiguously.

Chomsky Normal Form:
• A context – free grammar is in Chomsky normal form, if every rule is of the form
o A → BC
o A→a
• where a is any terminal and A, B and C are any variables – except that B and C may not be the start
variable. In addition we permit the rule S → ε , where S is the start variable.
• Any context – free language is generated by a context – free grammar in Chomsky normal form.

Pushdown Automata (PDA):
• These automata are like NFA but have an extra component called a attack. The stack provides
additional memory beyond the finite amount available in the control. The stack allows pushdown
automata to recognize some Nonregular languages.
• Pushdown automata are equivalent in power to context – free grammars.
• A stack is valuable because it can hold an unlimited amount of information.
• The current state, the next input symbol read and the top symbol of the stack determine the next move
of a pushdown automaton.
• Definition: A pushdown automaton is a 6 – tuple (Q, Σ, Γ, δ , q 0 , F ) , where Q, Σ, Γ and F are all finite
sets, and:
1. Q is the set of states.
2. Σ is the input alphabet.
3. Γ is the stack alphabet.
4. δ : Q × Σ ε × Γε → P(Q × Γε ) is the transition function.
5. q 0 ∈ Q is the start state.
• We write a, b → c to signify that when the machine is reading an a from the input it may replace the
symbol b on the top of the stack with a c. Any of a, b and c may be ε. If a is ε, the machine may make
this transition without reading any symbol form the input. If b is ε, the machine may make this
transition without reading and popping any symbol from the stack. If c is ε, the machine does not write
any symbol on the stack when going along this transition.

Equivalence with context – free grammars:
• A language is context free if and only if some pushdown automaton recognizes it.
• Constructing a CFG out of a PDA, p. 108 – 110.
• Every regular language is context – free.

6


Pumping lemma for context – free languages:
• If A is a context – free language, then there is a number p (the pumping length) where, if s is any
string in A of length at least p, then s may be divided into five pieces s = uvxyz satisfying the
conditions:
1. For each i ≥ 0, uv i xy i z ∈ A
2. | vy | > 0
3. | vxy | ≤ p

Chapter 3: The Church – Turing Thesis
Turing Machines:
• Similar to a finite automaton but with an unlimited and unrestricted memory, a Turing machine is a
much more accurate model of a general purpose computer. A Turing machine can do everything that a
real computer can do. Nonetheless, even a Turing machine cannot solve certain problems. In a very
real sense, these problems are beyond the theoretical limits of computation.
• The following list summarizes the differences between finite automata and Turning machines:
1. A Turing machine can both write on the tape and read from it.
2. The read – write head can move both to the left and to the right.
3. The tape is infinite.
4. The special states for rejecting and accepting take immediate effect.
• In actuality we almost never give formal descriptions of Turing machines because they tend to be very
big.
• Definition: A Turing machine is a 7 – tuple (Q, Σ, Γ, δ , q 0 , q accept , q reject ) , where Q, Σ, Γ are all finite
sets and:
1. Q is the set of states.
2. Σ is the input alphabet not containing the special blank symbol
3. Γ is the tape alphabet, where ∈ Γ and Σ ⊆ Γ
4. δ : Q × Γ → Q × Γ × {L, R} is the transition function
5. q 0 ∈ Q is the start state
6. q accept ∈ Q is the accept state
7. q reject ∈ Q is the reject state, where q reject ≠ q accept $
• For a Turing machine, δ takes the form: δ : Q × Γ → Q × Γ × {L, R} . That is, when the machine is in a
certain state q1 and the head is over a tape square containing a symbol a, and if δ (q1 , a ) = (q 2 , b, L) ,
the machine writes the symbol b replacing the a, and goes to state q 2 .
• Initially M receives its input w = w1 w2 ...wn ∈ Σ * on the leftmost n squares of the tape, and the rest of
the tape is blank.
• As a Turing machine computes, changes occur in the current state, the current tape contents, and the
current head location. A setting of these three items is called a configuration of the Turing machine.
• A Turing machine M accepts input w if a sequence of configurations C1 , C 2 ,..., C k exists where
1. C1 is the start configuration of M on input w
2. Each C i yields C i +1
3. C k is an accepting configuration.
• The collection of strings that M accepts is the language of M, denoted L(M).
• Call a language Turing – recognizable if some Turing machine recognizes it.
• When we start a TM on an input, three outcomes are possible. The machine may accept, reject, or
loop. By loop we mean that the machine simply does not halt. It is not necessarily repeating the same
steps in the same way forever as the connotation of looping may suggest. Looping may entail any
simple or complex behaviour that never leads to a halting state.

7


• We prefer Turing machines that halt on all inputs; such machines never loop. These machines are
called deciders because they always make a decision to accept or reject. A decider that recognizes
some language also is said to decide that language.
• Call a language Turing – decidable or simply decidable if some Turing machine decides it.
• Every decidable language is Turing – recognizable but certain Turing – recognizable languages are not
decidable.

Variants of Turing Machines:
• The original TM model and its reasonable variants all have the same power – they recognize the same
class of languages.
• To show that two models are equivalent we simply need to show that we can simulate one by the
other.
• A multitape TM is like an ordinary Turing machine with several tapes. Each tape has its own head for
reading and writing. Initially the input appears on tape 1, and the others start out blank.
• Two machines are equivalent if they recognize the same language.
• Every multitape Turing machine has an equivalent single tape Turing machine.
• A language is Turing – recognizable if and only if some multitape Turing machine recognizes it.
• A nondeterministic Turing machine is defined in the expected way. At any point in a computation the
machine may proceed according to several possibilities. The transition function for a nondeterministic
Turing machine has the form:
o δ : Q × Γ → P(Q × Γ × {L, R})
• The computation of a nondeterministic Turing machine is a tree whose branches correspond to
different possibilities for the machine. (If you want to simulate a nondeterministic TM with a “normal”
TM you have to perform a breadth – first search through the tree, because with depth – first you can
lose yourself in a infinite branch of the tree and miss the accept state). If some branch of the
computation leads to the accept state, the machine accepts its input.
• Every nondeterministic Turing machine has an equivalent deterministic Turing machine.
• A language is Turing – recognizable if and only if some nondeterministic Turing machine recognizes
it.
• We call a nondeterministic Turing machine a decider if all branches halt on all inputs.
• A language is decidable if and only if some nondeterministic TM decides it.
• Loosely defined, an enumerator is a Turing machine with an attached printer.
• A language is Turing – recognizable if and only if some enumerator enumerates it.

The Definition of Algorithm:
• Informally speaking, an algorithm is a collection of simple instructions for carrying out some task.
• Alonzo Church used a notational system called the λ - calculus to define algorithms. Turing did it with
his “machines”. These two definitions were shown to be equivalent. This connection between the
informal notion of algorithm and the precise definition has come to be called the Church – Turing
thesis.
• The Church – Turing thesis:
o Intuitive notion of algorithm is equal to Turing machine algorithms.
• Our notation for the encoding of an object O into its representation as a string is <O>. If we have
several objects O1 , O2 ,..., Ok we denote their encoding into a single string by < O1 , O2 ,..., Ok > .
• Example on page. 145.

8


Chapter 4: Decidability
Decidable Languages:
• Acceptance problem expressed as languages for regular expressions:
o ADFA = {< B, w > | B is a DFA that accepts input string w}
o The problem of testing whether a DFA B accepts an input w is the same as the problem of
testing whether <B,w> is a member of the language ADFA . Similarly, we can formulate other
computational problems in term of testing membership in a language. Showing that the
language is decidable is the same as showing that the computational problem is decidable.
o ADFA is a decidable language.
o ANFA = {< B, w > | B is an NFA that accepts input string w}
o ANFA is a decidable language.
o AREX = {< R, w > | R is a regular expression that generates string w }
o AREX is a decidable language.
• Emptiness testing for regular expressions:
o E DFA = {< A > | A is a DFA and L(A) = ∅ }
o This means, that no string exists that DFA A accepts.
o E DFA is a decidable language.
• The next theorem states that testing whether two DFAs recognize the same language is decidable:
o EQDFA = {< A, B > | A and B are DFAs and L(A) = L(B) }
o EQDFA is a decidable language.
• Acceptance problem expressed as languages for context – free languages:
o ACFG = {< G , w > | G is a CFG that generates string w }
o ACFG is a decidable language.
o The problem of testing whether a CFG generates a particular string is related to the problem of
compiling programming languages.
• Emptiness testing for context – free grammars:
o ECFG = {< G > | G is a CFG and L(G) = ∅ }
o ECFG is a decidable language.
• Equivalence testing for context – free grammars:
o EQCFG = {< G , H > | G and H are CFLs and L(G) = L(H) }
o EQCFG is not decidable
• Every context – free language is decidable.

The Halting Problem:
• ATM = {< M , w > | M is a TM and M accepts w }
• ATM is undecidable but ATM is Turing – recognizable hence ATM is sometimes called the halting
problem.

The Digitalisation Method:
• Cantor observed that two finite sets have the same size if the elements of one set can be paired with
the elements of the other set.
• Assume A and B are two (infinite) sets. If then exists a bijektive function f between the two sets, they
have the same size.
• A set B is countable if either it is finite or it has the same size as the natural numbers N.
• Q (rational numbers) and N have the same size.
• R (real numbers) is uncountable.

9


• It shows that some languages are not decidable or even Turing – recognizable, for the reason that there
are uncountable many languages yet only countably many Turing machines. Because each Turing
machine can recognize a single language and there are more languages than Turing machines, some
languages are not recognizable by any Turing machine.
• Some languages are not Turing – recognizable.
• The following theorem shows that, if both a language and its complement are Truing – recognizable,
the language is decidable. Hence, for any undecidable language, either it or its complement is not
Truing – recognizable. We say that a language is co – Turing – recognizable if it is the complement of
a Turing – recognizable language.
• A language is decidable if and only if it is both Turing – recognizable and co – Turing – recognizable.
• ATM is not Turing – recognizable.

Chapter 5: Reducibility
Undecidable Problems from Language Theory:
• In this chapter we examine several additional unsolvable problems. In doing so we introduce the
primary method for proving that problems are computationally unsolvable. It is called reducibility.
• A reduction is a way of converting gone problem into another problem I such a way that a solution to
the second problem can be used to solve the first problem.
• When A is reducible to B, solving A cannot be harder than solving B because a solution to B gives a
solution to A. In terms of computability theory, if A is reducible to B and B is decidable, A also is
decidable. Equivalently, if A is undecidable and reducible to B, B is undecidable. This last version is
key to proving that various problems are undecidable.
• HALTTM = {< M , w > | M is a TM and M halts on input w }
• HALTTM is undecidable
• ETM = {< M > | M is a TM and L(M) = ∅ }
• ETM is undecidable.
• REGULARTM = {< M > | M is a TM and L(M) is a regular language }
• REGULARTM is undecidable.
• Rice’s theorem:
o Testing any property of the languages recognized by a TM is undecidable.
• EQTM = {< M 1 , M 2 > | M 1 and M 2 are TMs and L( M 1 ) = L( M 2 ) }
• EQTM is undecidable.
• Despite their memory constraints, linear bounded automata (LBA) are quite powerful. For example,
the deciders for ADFA , ACFG , E DFA , ECFG all are LBAs. Every CFL can be decided by an LBA.
• Let M be an LBA with q states and g symbols in the tape alphabet. There are exactly q ⋅ n ⋅ g n distinct
configurations of M for a tape of length n.
• ALBA = {< M , w > | M is a LBA that accepts string w }
• ALBA is decidable.
• E LBA = {< M > | M is an LBA where L(M) = ∅ }
• E LBA is undecidable.
• ALLCFG = {< G > | G is a CFG and L(G ) = Σ * }
• ALLCFG is undecidable.
• This is the problem of testing whether a context – free grammar generates all possible strings.

Mapping Reducibility:
• Roughly speaking, being able to reduce problem A to problem by using a mapping reducibility means
that a computable function exists that converts instances of problem A to instances of problem B. If
we have such a conversion function, called a reduction, we can solve a with a solver for B.

10


• A function f : Σ* → Σ * is a computable function if some Turing machine M, on every input w, halts
with just f(w) on its tape.
• Language A is mapping reducible to language B, written A ≤ m B , if there is a computable function
f : Σ* → Σ * , where for every w,
o w ∈ A ⇔ f ( w) ∈ B
• The function f is called the reduction of A to B.
• If A ≤ m B and B is decidable, then A is decidable.
• If A ≤ m B and A is undecidable, then B is undecidable.
• If A ≤ m B and B is Turing – recognizable, then A is Turing – recognizable.
• If A ≤ m B and A is not Turing – recognizable, then B is not Turing – recognizable.

Chapter 7: Time Complexity

Measuring Complexity:
• Even when a problem is decidable and thus computationally solvable in principle, it may not be
solvable in practice if the solution requires an inordinate amount of time or memory. In this final part
of the book we introduce computational complexity theory – an investigation of the time, memory, or
other resources required for solving computational problems.
• For simplicity we compute the running time of an algorithm purely as a function of the length of the
string representing the input and don’t consider any other parameters. In worst – case analysis, the
form we consider here, we consider the longest running time of all inputs of a particular length.
• Definition:
o Let M be a deterministic Turing machine that halts on all inputs. The running time or time
complexity of M is the function f : N → N , where f (n) is the maximum number of steps that
M uses on any input of length n. If f (n) is the running time of M, we say that M runs in time
f (n) and that M is an f (n) time Turing machine.
• Because the exact running time of an algorithm often is a complex expression, we usually just estimate
is. In one convenient form of estimation, called asymptotic analysis, we seek to understand the running
time of the algorithm when it is run on large inputs.
• Definition:
o Let f and g be to functions f , g : N → R + . Say that f (n) = O( g (n)) if positive integers c and
n0 exist so that for every integer n ≥ n0
§ f ( n ) ≤ c ⋅ g ( n)
o When f (n) = O( g (n)) we say that g(n) is an upper bound for f(n), or more precisely, that g(n)
is an asymptotic upper bound for f(n), to emphasize that we are suppressing constant factors.
• Frequently we derive bounds of the form n c for c greater than 0. Such bounds are called polynomial
δ
bounds. Bounds of the form 2 ( n ) are called exponential bounds when δ is a real number greater than
0.
• Big – O notation has a companion called small – o notation. Big – O notation gives a way to say that
one function is asymptotically no more than another. To say that one function is asymptotically less
than another we use small – o notation. The difference between the big – O and small – o notation is
analogous to the difference between ≤ and <.
• Definition:
o Let f and g be two functions f , g : N → R + . Say that f (n) = o( g (n)) if
f (n)
§ lim =0
n → ∞ g ( n)

o In other words, f (n) = o( g (n)) means that, for any real number c > 0, a number n0 exists,
where f (n) < c ⋅ g (n) for all n ≥ n0 .

11


• Definition:
o Let t : N → N be a function. Define the time complexity class, TIME(t(n)), to be
§ TIME(t(n)) = { L | L is a language decided by an O(t(n)) time Turing machine }
• Any language that can be decided in o(n ⋅ log n) time on a single – tape Turing machine is regular.
• This discussion highlights an important difference between complexity theory and computably theory.
In computability theory, the Church – Turing thesis implies that all reasonable models of computation
are equivalent, that is, they all decide the same class of languages. In complexity theory, the choice of
the model affects the time complexity of languages.

Complexity Relationships among Models:
• Let t(n) be a function, where t (n) ≥ n . Then every t(n) time multitape Turing machine has an
equivalent O (t 2 (n)) time single – tape Turing machine.
• Definition:
o Let N be a nondeterministic Turing machine that is a decider. The running time of N is the
function f : N → N , where f(n) is the maximum number of steps that N uses on any branch
of its computation on any input of length n.
• Let t(n) be a function, where t (n) ≥ n . Then every t(n) time nondeterministic single – tape Turing
machine has an equivalent 2 O (t ( n )) time deterministic single tape Turing machine.

The Class P:
• Exponential time algorithms typically arise when we solve problems by searching through a space of
solutions, called brute – force search.
• P is the class of languages that are decidable in polynomial time on a deterministic single – tape
Turing machine. In other words:
o P = U TIME (n k )
k
• The class P plays a central role in our theory and is important because
o P is invariant for all models of computation that are polynomially equivalent to the
deterministic single – tape Turing machine.
o P roughly corresponds to the class of problems that are realistically solvable on a computer.
• Every context – free language is a member of P.

The Class NP:
• Hamilton – Path: HAMPATH = {<G,s,t> | G is a directed graph with a Hamiltonian path from s to t}
• The HAMPATH problem does have a feature called polynomial verifiability.
• Some problems may not be polynomial verifiable. For example, take HAMPATH , the complement of
the HAMPATH problem. Even if we could determine (somehow) that a graph did not have a
Hamiltonian path, we don’t know of a way for someone else to verify its non-existence without using
the same exponential time algorithm for making the determination in the first place.
• A verifier for a language A is an algorithm V, where
o A = {w | V accepts <w,c> for some string c }
• We measure the time of a verifier only in terms of the length of w, so a polynomial time verifier runs
in polynomial time in the length of w.
• A verifier uses additional information, represented by the symbol c, to verify that a string w is a
member of A. This information is called a certificate, or proof, of membership in A. Observe that, for
polynomial verifiers, the certificate has polynomial length (in the length of w) because that is all the
verifier can access in its time bound.
• NP is the class of languages that have polynomial time verifiers.
• A language is in NP iff it is decided by some nondeterministic polynomial time Turing machine.
• NTIME(t(n)) = { L | L is a language decided by a O(t(n)) time nondeterministic Turing machine }
• NP = U NTIME (n k )
k

12


• CLIQUE = { <G,k> | G is an undirected graph with a k – clique }
• CLIQUE is in NP.
• SUBSET-SUM = { <S,t> | S = {x1 ,..., x k } and for some { y1 ,..., y l } ⊆ {x1 ,..., x k } , we have ∑y i =t}
• SUBSET-SUM is in NP.
• Observe that the complements of these stets, CLIQUE and SUBSET − SUM , are not obviously
members of NP. Verifying that something is not present seems to be more difficult than verifying that
it is present. We make a separate complexity class, called coNP, which contains the languages that are
complements of languages in NP. We don’t know whether coNP is different form NP.

NP – Completeness:
• SAT = { <Φ> | Φ is a satisfiable Boolean formula }
• Cook – Levin theorem: SAT ∈ P ⇔ P = NP
• In Chapter 5, we defined the concept of reducing one problem to another. When problem A reduces to
problem B, a solution to B can be used to solve A. Now we define a version of reducibility that takes
the efficiency of computation into account. When problem A is efficiently reducible to problem B, an
efficient solution to B can be used to solve A efficiently.
• Definition:
o A function f : Σ* → Σ * is a polynomial time computable function if some polynomial time
Turing machine M exists that halts with just f(w) on its tape, when started on any input w.
• Definition:
o Language A is polynomial time mapping reducible, or simply polynomial time reducible, to
language B, written A ≤ p B , if a polynomial time computable function f : Σ* → Σ * exists,
where for every w,
§ w ∈ A ⇔ f ( w) ∈ B
o The function f is called the polynomial time reduction of A to B.
• If A ≤ p B and B ∈ P , then A ∈ P .
• 3SAT = { <Φ> | Φ is a satisfiable 3-CNF-formula }
• 3SAT is polynomial time reducible to CLIQUE. This means, if CLIQUE is solvable in polynomial
time, so is 3SAT.
• Definition:
o A language B is NP – complete if it is satisfies two conditions:
1. B is in NP
2. Every A in NP is polynomial time reducible to B.
• If B is NP – complete and B ∈ P , then P = NP.
• If B is NP – complete and B ≤ p C for C in NP, then C is NP – complete.
• SAT is NP – complete.
• 3SAT is NP – complete.
• CLIQUE is NP – complete.
• If G is an undirected graph, a vertex cover of G is a subset of the nodes where every edge of G touches
one of those nodes. The vertex cover problem asks for the size of the smallest vertex cover.
• VERTEX-COVER = { <G,k> | G is an undirected graph that has a k – node vertex cover }
• VERTEX-COVER is NP – complete.
• HAMPATH is NP – complete.
• SUBSET-SUM is NP – complete.

13


Introduction to the theory of computation

More Related Content

What's hot (20)

Similar to Introduction to the theory of computation (20)

More from prasadmvreddy (20)

Recently uploaded (20)

Introduction to the theory of computation