Title

ECS 120 Theory of Computation
Turing Machines
Julian Panetta
University of California, Davis

Finishing Unit 1: Proof of the Myhill-Nerode Corollary

Corollary (one direction) of the Myhill-Nerode Theorem:
If a language \(L\) has an infinite number of equivalence classes with respect to \(\sim_L\),
then \(L\) is nonregular.

Proof: (Contrapositive: \(L\) regular \(\implies\) \(\sim_L\) defines a finite number of equivalence classes.)

  • Let \(L\) be regular, and let \(D = (Q, \Sigma, \delta, s, F)\) be a DFA deciding \(L\).
  • Denote by \(\reachedState(x)\) the state reached by \(D\) after processing a string \(x \in \Sigma^*\).
    • \(\reachedState(x)\) is sometimes called the “extended transition function.”
    • It is defined recursively as:
      • Base case: \(\; \reachedState(\emptystring) = \fragment{s}\)
      • For all \(a \in \Sigma\) and \(x \in \Sigma^*\): \(\; \reachedState(x a) = \fragment{\delta(\reachedState(x), a)}\)
  • For any \(x, y \in \Sigma^*\) such that \(\reachedState(x) = \reachedState(y)\), we have \(x \sim_L y \quad\) (\(\reachedState(xz) = \reachedState(yz)\) for all \(z \in \Sigma^*\)).
  • Thus for each state \(q \in Q\), all strings \(x \in \Sigma^*\) with \(\reachedState(x) = q\) are in the same equivalence class.
  • This means the number of equivalence classes defined by \(\sim_L\) is at most \(|Q|\) (hence finite).

Models of computation

  • Models deciding regular languages:
    What can be done using constant memory and a single step per input symbol?
    • Finite Automata (DFA, NFA)
    • Regular Grammars (RRG, LRG)
    • Regular Expressions
  • Models deciding context-free languages:
    What can be done using a NFA with access to an unbounded stack?
    • Pushdown Automata (PDA) (Not covered; Section 2.2 of Sipser)
    • Equivalent in power to CFGs!

    Is this all we need?

  • Simple non-context-free languages:
    \[ L_1 = \setbuild{0^n 1^n 2^n}{n \in \mathbb{N}} = \{\epsilon, 012, 001122, \ldots\}, \quad \quad \fragment{L_2 = \setbuild{w w}{w \in \binary^*}} \]
    • These can be proved non-CFG-decidable using the pumping lemma for context-free languages
      (Optional Section 7.10 of lecture notes)

Models of computation

  • While DFAs and CFGs can solve many useful problems, they are quite limited.

  • Source of these limitations: lacking access to arbitrary amounts of memory.

    • (More precisely, pushdown automata have arbitrary memory but not arbitrary access to it. )

  • We now take a big leap in computational power and introduce Turing machines (TM).

    • These can do anything that a real computer can!
    • We will prove there are still problems that cannot be solved by TMs!

  • TMs are a generalization of DFAs.

    0 1 1 0 0,1 DFA q3 Control Input Tape Read head q3 b6725438-3a54-4bdc-8097-c64010750f74 q1 b6725438-3a54-4bdc-8097-c64010750f74->q1 q2 Read/write head Unbounded tape! ⎵ ⎵

This is now a Turing machine!

Turing machine (TM)

data/images/tm/dfa_as_tm.svg

Distinctive features of Turing machines:

  1. They can read and write symbols on the tape.
  2. The tape has symbols outside the input alphabet (e.g., the blank symbol “⎵”).
  3. The read/write head can move right, left, or stay in place according to the control logic.
  4. The tape is unbounded to the right and input starts at the leftmost cell.
    • If the head moves to the right of the current rightmost cell, a new “⎵” cell is created.
    • Sipser says “the tape is infinite,” but a more realistic mental model is a growable array
      (e.g., Python list, C++ std::vector, Java java.util.List).
  5. They have exactly one accept state and one reject state
    and halt immediately when either is reached.

Turing machine (TM)

data/images/tm/dfa_as_tm.svg

  • Once we gain intuition for TMs,
    we can describe the machine solving
    a given problem at a high-level while still being precise.
  • Example: design a TM to decide \(P = \setbuild{w \in \{0, 1\}^*}{w = \reverse{w}}\)
  • The TM will:
    1. “Zig-zag” back and forth between the left and right ends of the input, checking if the leftmost symbol equals the rightmost symbol. If not, reject immediately.
    2. “Cross off” symbols as they are checked by overwriting them with an \(\string{x}\) symbol.
    3. Continue until the entire input is crossed off, at which point the TM accepts.

Turing machine example (demo in simulator)

\[ P = \setbuild{w \in \{0, 1\}^*}{w = \reverse{w}} \]

  1. “Zig-zag” back and forth between the left and right ends of the input, checking if the leftmost symbol equals the rightmost symbol.
    If not, reject immediately.
  2. “Cross off” symbols as they are checked by overwriting them with an \(\string{x}\) symbol.
  3. Continue until the entire input is crossed off, at which point the TM accepts.

Formal definition of a Turing machine

A Turing machine (TM) is a 7-tuple: \(M = (Q, \Sigma, \Gamma, \delta, s, q_a, q_r)\) where:

  1. \(Q\) is a finite set of states.
  2. \(\Sigma\) is a finite set of input symbols (the input alphabet).
  3. \(\Gamma\) is a finite set of tape symbols (the tape alphabet) with:
    \(\Sigma \subsetneq \Gamma\) and blank symbol ⎵ \(\in \Gamma \setminus \Sigma\).
  4. \(s \in Q\) is the start state.
  5. \(q_a \in Q\) is the accept state.
  6. \(q_r \in Q\) is the reject state, where \(q_r \ne q_a\).
  7. \(\delta: \fragment{(Q \fragment{\setminus \{q_a, q_r\}}) \times \Gamma} \to \fragment{Q \times \fragment{\Gamma \times} \fragment{\{L, R, S\}}}\) is the transition function
    \(L\) = move left, \(R\) = move right, \(S\) = stay in place.

Formal definition of a Turing machine: Example

\[ P = \setbuild{w \in \{0, 1\}^*}{w = \reverse{w}} \]

data/images/tm/palindrome_tm/diagram.svg

\(M = (Q, \Sigma, \Gamma, \delta, s, q_a, q_r)\) where:

  • \(Q = \{s, r_{00}, r_{11}, r_{01}, r_{10}, \string{lx}, \string{l}, q_A, q_R\}\)
  • \(\Sigma = \{0, 1\}\)
  • \(\Gamma = \{0, 1, \string{x}, ⎵\}\)
  • \(s\) is the start state, \(q_A\) is the accept state,
    and \(q_R\) is the reject state
  • \(\delta\) is defined by the arrows in the diagram…

Formal definition of computation for Turing machines

How can we describe the “full state” that a Turing machine is in (including its tape)?

data/images/tm/dfa_as_tm.svg

A configuration of a TM \(M = (Q, \Sigma, \Gamma, \delta, s, q_a, q_r)\) is a triple: \(C = (q, p, w)\) where:

  • \(q \in Q\) is the current state of the TM.
  • \(p \in \N^+\) is the tape head position.
  • \(w \in \Gamma^\infty\) is a one-way infinite sequence holding the tape content

Sipser uses a different notation “\(u\, q\, v\)” where \(w=uv\) and \(p = |u| + 1\).
Example: \(1011q_70111\).

See Optional Section 8.4 for a definition (uglier) using a finite, growable tape.

Formal definition of computation for Turing machines

data/images/tm/dfa_as_tm.svg

A configuration of a TM \(M = (Q, \Sigma, \Gamma, \delta, s, q_a, q_r)\) is a triple: \(C = (q, p, w)\) where:

  • \(q \in Q\) is the current state of the TM.
  • \(p \in \N^+\) is the tape head position.
  • \(w \in \Gamma^\infty\) is a one-way infinite sequence holding the tape content

  • We define computation by formally specifying how one configuration \(C = (q, p, w)\) transitions to the next configuration \(C' = (q', p', w')\).

  • We say \(C\) yields \(C'\) (writing \(C \to C'\)) when \(M\) can legally go from \(C\) to \(C'\) in one step: \[ \delta(q, w[p]) = (\fragment{q'}, \fragment{w'[p]}, \fragment{m}) \hspace{5em} \]

    We identify moves \(m \in \{L, R, S\}\) with offsets \(\{-1, +1, 0\}\) added to \(p\).

    • \(p' = \max(1, p + m)\)
      (Move tape head according to \(m\) but don’t go off the left end.)
    • \(w[i] = w'[i]\) for all \(i \ne p\)
      (Leave all other tape cells unchanged.)

Formal definition of computation for Turing machines

data/images/tm/dfa_as_tm.svg

A configuration of a TM \(M = (Q, \Sigma, \Gamma, \delta, s, q_a, q_r)\) is a triple: \(C = (q, p, w)\) where:

  • \(q \in Q\) is the current state of the TM.
  • \(p \in \N^+\) is the tape head position.
  • \(w \in \Gamma^\infty\) is a one-way infinite sequence holding the tape content

  • A configuration \((q, p, w)\) is called:
    accepting if \(q = q_a\), rejecting if \(q = q_r\), halting if \(q \in \{q_a, q_r\}\).

  • \(M\) accepts/rejects input \(w \in \Sigma^*\) if there is a finite sequence of configurations \(C_1, C_2, \ldots, C_k\) such that:

    1. \(C_1 = (\fragment{s}, \fragment{1}, \fragment{w \string{⎵}^\infty})\)
    2. \(C_i \to C_{i+1}\) for all \(1 \leq i < k\)
    3. \(C_k\) is accepting/rejecting

  • If \(M\) neither accepts nor rejects \(w\), we say \(M\) loops (does not halt) on \(w\).

  • Three possible outcomes: accept, reject, or loop!
  • Defining the languages “recognized” or “decided” by a TM requires more care than with FAs or CFGs.