ECS 120 Theory of Computation

Nondeterministic Finite Automata

Julian Panetta

University of California, Davis

Reminder: Operation of an NFA

Intuitively, upon reading each symbol, an NFA follows all possible paths.
- We can think of this as the machine “forking” into multiple copies to take each available transition.
- For each copy, if there is no transition for the consumed symbol, that copy “dies”.
The NFA accepts the input string if any of its “copies” is in an accept state after reading the entire input string. Otherwise, it rejects.

Example processing the string \(010110\):

data/images/nfa/n1/computation_tree_010110_animated.svg

← Accept!

Operation of an NFA

A simpler way to simulate the NFA: keep track of the set of states that are “active” after reading each symbol.

Example processing the string 010110:

010110

Begin in the state(s) \(\{q_1\}\).
Read 0 and transition to state(s) \(\{q_1\}\).

010110
Read 1 and transition to state(s) \(\{q_1, q_2, q_3\}\).

010110
Read 0 and transition to state(s) \(\{q_1, q_3\}\).

010110
Read 1 and transition to state(s) \(\{q_1, q_2, q_3, q_4\}\).

010110
Read 1 and transition to state(s) \(\{q_1, q_2, q_3, q_4\}\).

010110
Read 0 and transition to state(s) \(\{q_1, q_3, q_4\}\).

010110
Since an accept state is highlighted, the NFA accepts the string.

Operation of an NFA

A simpler way to simulate the NFA: keep track of the set of states that are “active” after reading each symbol.

Which of the following strings are accepted by this NFA?

This NFA decides the language: \[\setbuild{w \in \{0, 1\}^*}{w \text{ has 101 or 11 as a substring }}\]

Example NFAs

Let’s design an NFA that accepts binary strings with a \(1\) in the third-to-last position: \(\setbuild{w \in \{0, 1\}^*}{w[|w| - 2] = 1}\).

DFA for same language?

Example NFAs

What language over the alphabet \(\{0, 1\}\) does this NFA accept?

\(\emptyset\)
\(\{\emptystring\}\)
\(\{0, 1\}^*\)

Formal Definition of an NFA

Key differences from DFAs that we must encode:

Any number of transitions (including 0) for the same state and input symbol.
Make the transition function output a set of states!
The possibility of \(\emptystring\)-transitions, which do not consume any input symbols.
Include \(\emptystring\) in the set of possible second arguments to transition function: \(\Sigma_\emptystring \defeq \Sigma \cup \{\emptystring\}\)

A nondeterministic finite automaton (NFA) is a 5-tuple \((Q, \Sigma, \Delta, q_0, F)\) where:

\(Q\) is a finite set of states.
\(\Sigma\) is a finite set of symbols called the alphabet.
\(\Delta: \fragment{Q \times} \fragment{\Sigma_\emptystring \to} \fragment{\powerset(Q)}\) is the transition function.
\(q_0 \in Q\) is the start state.
\(F \subseteq Q\) is the set of accept states.

For convenience, we can assume that if \(\Delta(q, a)\) is not explicitly defined for \(q \in Q\) and \(a \in \Sigma\), then \(\Delta(q, a) = \emptyset\).

Translating NFA State Diagrams to Formal Definitions

\(Q = \{q_1, q_2, q_3, q_4\}\)
\(\Sigma = \{0, 1\}\)

\(\Delta\) is defined by the following lookup table:

	\(0\)	\(1\)	\(\emptystring\)
\(q_1\)	\(\fragment{\{q_1\}}\)	\(\fragment{\{q_1, q_2\}}\)	\(\fragment{\emptyset}\)
\(q_2\)	\(\fragment{\{q_3\}}\)	\(\fragment{\emptyset}\)	\(\fragment{\{q_3\}}\)
\(q_3\)	\(\fragment{\emptyset}\)	\(\fragment{\{q_4\}}\)	\(\fragment{\emptyset}\)
\(q_4\)	\(\fragment{\{q_4\}}\)	\(\fragment{\{q_4\}}\)	\(\fragment{\emptyset}\)

\(q_1\) is the start state.
\(F = \{q_4\}\)

Alternative Specification of an NFA: Sets of Transitions

We can equivalently formalize the arrows of the NFA state diagram as a set of transitions.
This is similar to how we defined the productions of a CFG as a set of rules.
This NFA has the transitions:
\[ \{ q_1 \stackrel{0}{\to} q_1, \; q_1 \stackrel{1}{\to} q_1, \; q_1 \stackrel{1}{\to} q_2, \; q_2 \stackrel{0}{\to} q_3, \; q_2 \stackrel{\emptystring}{\to} q_3,\; q_3 \stackrel{1}{\to} q_4,\; q_4 \stackrel{0}{\to} q_4,\; q_4 \stackrel{1}{\to} q_4 \} \]
Each of these transitions is formally an ordered triple \((q_i, a, q_j) \in Q \times \Sigma_\emptystring \times Q\) where \(q_i\in Q\) is the source state, \(a \in \Sigma_\emptystring\) is the symbol read, and \(q_j \in Q\) is the destination state.
We can therefore specify the NFA’s transitions as some subset of \(Q \times \Sigma_\emptystring \times Q\).
This formulation makes it easy to talk about adding or removing transitions in the next chapter when we dealing with “constructions”: algorithmic specification of automata.

Formal Definition of NFA Computation

The finite automaton \(N = (Q, \Sigma, \Delta, q_0, F)\) accepts a string \(w\)
if we can write \(w\) as \(w= y_1 y_2 \ldots y_n \in \Sigma_\emptystring^n\); note possibly \(n>|w|\) since some \(y_i\) may be \(\varepsilon\)
and there exists a sequence of states \(r_0, r_1, \ldots, r_n \in Q\) such that:

\(r_0 = q_0\)
\(r_n \in F\)
\(r_i \in \Delta(r_{i-1}, y_i)\) for all \(1 \leq i \leq n\)

We refer to the sequence \(r_0, r_1, \ldots, r_n\),
or the sequence of transitions followed, as a computation sequence of \(N\) on input \(w\).

\(N\) decides a language \(A\) (denoted \(L(N) = A\)) if \(A = \setbuild{w \in \Sigma^*}{N \text{ accepts } w}\).

A language/decision problem \(A\) is NFA-decidable if there is an NFA \(N\) such that \(L(N) = A\).

Closure of Language Classes

Our goal in studying models of computation is not to become experts in designing automata, regular expressions, or grammars, but rather to understand these models’ computational power.
A useful tool for understanding this is the concept of closure.
- Recall that we say the natural numbers are closed under addition because for all \(a, b \in \mathbb{N}\), also \(a + b \in \mathbb{N}\) (i.e., the sum of two natural numbers is also a natural number).
  - Note the natural numbers are not closed under subtraction since \(3 - 5 \notin \mathbb{N}\).
- For languages, we will consider the operations:
  - Complement: \(\overline{L} = \Sigma^* \setminus L = \setbuild{w \in \Sigma^*}{w \notin L}\)
  - Union: \(L_1 \cup L_2 = \setbuild{w \in \Sigma^*}{w \in L_1 \text{ or } w \in L_2}\)
  - Intersection: \(L_1 \cap L_2 = \setbuild{w \in \Sigma^*}{w \in L_1 \text{ and } w \in L_2}\)
  - Concatenation: \(L_1 L_2 = \setbuild{w_1 w_2}{w_1 \in L_1 \text{ and } w_2 \in L_2}\)
  - Kleene Star: \(L^* = \setbuild{w_1 w_2 \ldots w_k}{k \geq 0 \text{ and } w_i \in L}\)
  Practically, if a class of languages is closed under \(\cup\), then knowing \(L_1\) and \(L_2\) are both in the class immediately proves \(L_1 \cup L_2\) is too.
  
  Or if you want to prove \(L\) is not DFA decidable, it might be easier to prove \(\overline{L}\) is not DFA decidable.

Closure of Language Classes

Strategy for proving closure: build constructions!
- A statement like “The class of DFA-decidable languages is closed under complement” is really an “existence theorem”.
- It means: for every DFA \(D\), deciding language \(L(D)\) there exists a DFA \(D'\) that decides \(\overline{L(D)}\).
- A “construction” is essential an algorithm for producing \(D'\), given \(D\).
In general, we prove the language class decidable by a certain model is closed under an operation “\(\texttt{op}\)” by specifying algorithms for converting instances of the model into a new instance of the same model deciding a new language. \[ L(D_1)\; \texttt{op}\; L(D_2) = L(\texttt{dfa\_op\_construction}(D_1, D_2)) \]
Later on, we will relate DFA-, NFA-, regex-, and CFG-decidable languages by showing how an instance from one model can be transformed into an instance of another model deciding the same language. Example: \[ L(N) = L(D) \quad \text{where} \quad D = \texttt{nfa\_to\_dfa\_construction}(N) \hspace{5em} \]

Closure of DFA-Decidable Languages under Complement

Recall the DFA \(D\) for \(L = \setbuild{w \in \{0, 1\}^*}{w \text{ contains the substring } 001}\):
How can we build, from \(D\), a DFA that decides \(\overline{L}\)?
Easy: just flip which states are accepting!
DFA deciding \(\overline{L} = \setbuild{w \in \{0, 1\}^*}{w \text{ does not contain } 001}\)

Theorem: The class of DFA-decidable languages is closed under complement.

Proof: For any language \(A\) in this class, there by definition exists a DFA \(\, D = (Q, \Sigma, \delta, s, F)\) such that \(A = L(D)\).

We can therefore construct the DFA \(\,\overline{D} = (Q, \Sigma, \delta, s, \fragment{Q \setminus F})\) that decides \(\overline{A}\) (so that \(\overline{A} = L(\overline{D})\)).

Upon processing a string \(w\), both DFAS end in state \(q \in Q\). So \(w \in \overline{A} \fragment{\iff w \notin A} \fragment{\iff q \notin F} \fragment{\iff q \in Q \setminus F.}\)

What about closure under complement of:

NFA-decidable languages?
True, but not obvious!
regex-decidable languages?
True, but not obvious!
CFG-decidable languages?
Actually false!

Regex Closure under Union, Concatenation, and Star

How can we prove that the class of regex-decidable languages is closed under:
- Union: \(L_1 \cup L_2 = \setbuild{w \in \Sigma^*}{w \in L_1 \text{ or } w \in L_2}\)
- Concatenation: \(L_1 L_2 = \setbuild{w_1 w_2}{w_1 \in L_1 \text{ and } w_2 \in L_2}\)
- Kleene Star: \(L^* = \setbuild{w_1 w_2 \ldots w_k}{k \geq 0 \text{ and } w_i \in L}\)
This is actually trivial due to the recursive regex definition!
What about intersection?
- We often can make ad-hoc arguments, but a general algorithm isn’t obvious.
- Example: let’s match strings over \(\Sigma = \{0, 1\}\) that contain \(0011\) and have an odd length. \[\begin{align*} R_1 &= \Sigma^* 0011 \Sigma^* \quad \color{gray} \text{Match strings containing 0011.} \\ R_2 &= \Sigma (\Sigma \Sigma)^* \quad \quad \color{gray} \text{Match odd-length strings.} \\ R_3 &= \Big(\Sigma (\Sigma \Sigma)^* 0011 (\Sigma \Sigma)^*\Big) \cup \Big((\Sigma \Sigma)^* 0011 \Sigma (\Sigma \Sigma)^*\Big) \hspace{5em} \end{align*}\]

There’s no obvious algorithm to compute \(R_3\), given arbitrary regex’s \(R_1\) and \(R_2\).

Certain properties will be easy or hard to prove for DFAs, regexes, or NFAs. But we will show the languages decided by each model are equivalent!

DFA Union

To gain practice with designing constructions, let’s prove the following

Theorem: the class of DFA-decidable languages is closed under union.
In other words, if \(L_1 = L(D_1)\) and \(L_2 = L(D_2)\), then we must prove the existence of a DFA \(D\) such that \(L_1 \cup L_2 = L(D)\).
- Idea: simulate \(D_1\) and \(D_2\) in parallel, and accept if either \(D_1\) or \(D_2\) accepts.
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string:
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: a
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaaaaa
- What do we need to keep track of to execute this simulation?
  Which pair of states are the machines in?

Title

Reminder: Operation of an NFA

Operation of an NFA

Operation of an NFA

Example NFAs

Example NFAs

Formal Definition of an NFA

Translating NFA State Diagrams to Formal Definitions

Alternative Specification of an NFA: Sets of Transitions

Formal Definition of NFA Computation

Closure of Language Classes

Closure of Language Classes

Closure of DFA-Decidable Languages under Complement

Regex Closure under Union, Concatenation, and Star

DFA Union