ECS 120 Theory of Computation

Closure of DFA-decidable languages

David Doty (adapted from slides authored by Julian Panetta)

University of California, Davis

Closure of language classes

Our goal in studying models of computation is not to become experts in designing automata, regular expressions, or grammars, but rather to understand these models’ computational power.
A useful tool for understanding this is the concept of closure.
- Recall that we say the natural numbers are closed under addition because for all \(a, b \in \mathbb{N}\), also \(a + b \in \mathbb{N}\) (i.e., the sum of two natural numbers is also a natural number).
  - Note the natural numbers are not closed under subtraction since \(3 - 5 \notin \mathbb{N}\).
- For languages, we will consider the operations:
  - Complement: \(\overline{L} = \Sigma^* \setminus L = \setbuild{x \in \Sigma^*}{x \notin L}\)
  - Union: \(A \cup B = \setbuild{x \in \Sigma^*}{x \in A \text{ or } x \in B}\)
  - Intersection: \(A \cap B = \setbuild{x \in \Sigma^*}{x \in A \text{ and } x \in B}\)
  - Concatenation: \(A B = \setbuild{x_1 x_2}{x_1 \in A \text{ and } x_2 \in B}\)
  - Kleene Star: \(L^* = \setbuild{x_1 x_2 \ldots x_k}{k \geq 0 \text{ and } x_i \in L}\)
  Practically, if a class of languages is closed under \(\cup\), then knowing \(A\) and \(B\) are both in the class immediately proves \(A \cup B\) is too.
  
  Or if you want to prove \(L\) is not DFA decidable, it might be easier to prove \(\overline{L}\) is not DFA decidable.

Roadmap for the next two chapters

Strategy for proving closure: build constructions!
- A statement like “The class of DFA-decidable languages is closed under complement” is really an “existence theorem”.
- It means: for every DFA \(D\), deciding language \(L(D)\) there is a DFA \(D'\) that decides \(\overline{L(D)}\).
- A “construction” is essential an algorithm for producing \(D'\), given \(D\).
In general, we prove the language class decidable by a certain model is closed under an operation “\(\texttt{op}\)” by specifying algorithms for converting instances of the model into a new instance of the same model deciding a new language. \[ L(D_1)\; \texttt{op}\; L(D_2) = L(\texttt{dfa\_op\_construction}(D_1, D_2)) \]
Later on, we will relate DFA-, NFA-, regex-, and CFG-decidable languages by showing how an instance from one model can be transformed into an instance of another model deciding the same language. Example: \[ L(N) = L(D) \quad \text{where} \quad D = \texttt{nfa\_to\_dfa\_construction}(N) \hspace{5em} \]

Closure of DFA-decidable languages under complement

Recall the DFA \(D\) for \(L = \setbuild{x \in \{0, 1\}^*}{x \text{ contains the substring } 001}\):
How can we build, from \(D\), a DFA that decides \(\overline{L}\)?
Easy: just flip which states are accepting! \(\Longrightarrow\) DFA deciding \(\overline{L} = \setbuild{x \in \{0, 1\}^*}{x \text{ does not contain } 001}\)

Theorem: The class of DFA-decidable languages is closed under complement.

Proof: For any language \(A\) in this class, there is a DFA \(\, D = (Q, \Sigma, \delta, s, F)\) such that \(A = L(D)\).

We can therefore construct the DFA \(\,D_c = (Q, \Sigma, \delta, s, \fragment{Q \setminus F})\) that decides \(\overline{A}\). Upon processing a string \(x\), both DFAs end in state \(q_x \in Q\). So \({x \in \overline{A} \fragment{\iff x \notin A} \fragment{\iff q_x \notin F} \fragment{\iff q_x \in Q \setminus F.}}\)

What about closure under complement of:

NFA-decidable languages?
True!
regex-decidable languages?
True!
CFG-decidable languages?
Actually false!

Complementing an NFA

True or False: Taking the complement of an NFA’s set of accept states makes it decide the complement of its original language.

True
False

data/images/nfa/nempty.svg \(\Longrightarrow\) data/images/nfa/complement/nempty_flipped.svg \(L(N) = \{\emptystring\}\) \(L(N_c) = \emptyset\) but \(\overline{L(N)} = \{0, 1\}^* \setminus \{\emptystring\}\)

data/images/nfa/complement/zeros.svg \(\Longrightarrow\) \(0 \in L(N)\) via \(s \transition{0} s\) \(0 \in L(N_c)\) via \(s \transition{\emptystring} q \transition{0} q\) but \(0 \notin \overline{L(N)}\)

What about closure under complement of:

NFA-decidable languages?
True, but not obvious!
regex-decidable languages?
True, but not obvious!
CFG-decidable languages?
Actually false!

Regex closure under union, concatenation, and star

How can we prove that the class of regex-decidable languages is closed under:
- Union: \(A \cup B = \setbuild{x \in \Sigma^*}{x \in A \text{ or } x \in B}\)
- Concatenation: \(A B = \setbuild{x_1 x_2}{x_1 \in A \text{ and } x_2 \in B}\)
- Kleene Star: \(L^* = \setbuild{x_1 x_2 \ldots x_k}{k \geq 0 \text{ and } x_i \in L}\)

This is actually trivial due to the recursive regex definition!
What about intersection?
- Example: let’s match strings over \(\Sigma = \{0, 1\}\) that contain \(0011\) and have an odd length. \[\begin{align*} R_1 &= \Sigma^* 0011 \Sigma^* \quad \color{gray} \text{Match strings containing 0011.} \\ R_2 &= \Sigma (\Sigma \Sigma)^* \quad \quad \color{gray} \text{Match odd-length strings.} \\ R_3 &= \require{mathtools} \underbrace{\Sigma (\Sigma \Sigma)^* 0011 (\Sigma \Sigma)^*}_{\mathclap{\genfrac{}{}{0pt}{0}{\text{odd-length strings with an odd-length prefix}} {\text{just before the substring } 0011}}} \fragment{{} \qquad \cup \qquad (\Sigma \Sigma)^* 0011 \Sigma (\Sigma \Sigma)^*} \hspace{5em} \end{align*}\]

There’s no obvious algorithm to compute \(R_3\), given arbitrary regex’s \(R_1\) and \(R_2\).

Certain closure properties will be easy or hard to prove for DFAs, regexes, or NFAs. But we will show the languages decided by each model are equivalent!

DFA union

To gain practice with designing constructions, let’s prove the following

Theorem: the class of DFA-decidable languages is closed under union.
In other words, if \(A = L(D_1)\) and \(B = L(D_2)\), then we must prove the existence of a DFA \(D\) such that \(A \cup B = L(D)\).
- Idea: simulate \(D_1\) and \(D_2\) in parallel, and accept if either \(D_1\) or \(D_2\) accepts.
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string:
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: a
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaaaa
  
  \(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) \(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
  Input string: aaaaaaaaaaaaaaa
- What do we need to keep track of to execute this simulation?
  Which pair of states are the machines in?

DFA union construction (product construction) example

Formally:

\(Q = \fragment{\{0, 1, 2\} \times \{0, 1, 2, 3, 4\}}\)
\(s = (0, 0)\)
\(\delta((i, j), \string{a}) = (\fragment{i + 1 \mod 3}, \fragment{j + 1 \mod 5})\)
\(F = \setbuild{(i, j)}{\fragment{i = 2 \text{ or } (j = 1 \text{ or } j = 4)}}\)

DFA union construction (product construction)

Let \(D_1 = (Q_1, \Sigma, \delta_1, s_1, F_1)\) and \(D_2 = (Q_2, \Sigma, \delta_2, s_2, F_2)\) be two DFAs.
We construct a new DFA \(D = (Q, \Sigma, \delta, s, F)\) to decide \(L(D_1) \cup L(D_2)\)
by simulating \(D_1\) and \(D_2\) in parallel:
- \(Q = Q_1 \times Q_2 = \setbuild{(p_1, p_2)}{p_1 \in Q_1 \text{ and } p_2 \in Q_2}\)
- \(s = \fragment{(s_1, s_2)}\)
- Define \(\delta\) for all \((p_1, p_2) \in Q\) and \(a \in \Sigma\) as: \(\quad\delta((p_1, p_2), a) = (\fragment{\delta_1(p_1, a)},\ \ \fragment{\delta_2(p_2, a)})\)
- \(F = \setbuild{(p_1, p_2)}{\fragment{p_1 \in F_1 \text{ or } p_2 \in F_2}}\)
Then after reading a string \(x\) that puts \(D_1\) in state \(p_1\) and \(D_2\) in state \(p_2\),
the new DFA \(D\) will be in state \((p_1, p_2)\).
- \((p_1, p_2) \in F\) if and only if \(p_1 \in F_1\) or \(p_2 \in F_2\).
  So \(x\) is accepted iff \(x \in L(D_1)\) or \(x \in L(D_2)\).
- Thus \(L(D) = L(D_1) \cup L(D_2)\).

Closure under intersection

We already showed that the DFA-decidable languages are closed under union:
If \(A = L(D_1)\) and \(B = L(D_2)\), then we can construct \(D\) such that \(L(D) = A \cup B\).

data/images/dfa/union/product-construction.svg

Formally:

Let \(D_1 = (Q_1, \Sigma, \delta_1, s_1, F_1)\) and
\(\phantom{\text{Let }} D_2 = (Q_2, \Sigma, \delta_2, s_2, F_2)\)
Construct \(D = (Q, \Sigma, \delta, s, F)\):
- \(Q = Q_1 \times Q_2 = \setbuild{(p_1, p_2)}{p_1 \in Q_1 \text{ and } p_2 \in Q_2}\)
- \(s = (s_1, s_2)\)
- For all \((p_1, p_2) \in Q\) and \(a \in \Sigma\): \[\delta((p_1, p_2), a) = (\delta_1(p_1, a), \delta_2(p_2, a))\]
- \[F = \setbuild{(p_1, p_2)}{p_1 \in F_1 \text{ or } p_2 \in F_2}\]

What about intersection?

Closure under intersection

Theorem: the DFA-decidable languages are closed under intersection:
If \(A = L(D_1)\) and \(B = L(D_2)\), then we can construct \(D\) such that \(L(D) = A \color{red} \cap \color{black} B\).

data/images/dfa/product-construction_intersection.svg

Formally:

Let \(D_1 = (Q_1, \Sigma, \delta_1, s_1, F_1)\) and
\(\phantom{\text{Let }} D_2 = (Q_2, \Sigma, \delta_2, s_2, F_2)\)
Construct \(D = (Q, \Sigma, \delta, s, F)\):
- \(Q = Q_1 \times Q_2 = \setbuild{(p_1, p_2)}{p_1 \in Q_1 \text{ and } p_2 \in Q_2}\)
- \(s = (s_1, s_2)\)
- For all \((p_1, p_2) \in Q\) and \(a \in \Sigma\): \[\delta((p_1, p_2), a) = (\delta_1(p_1, a), \delta_2(p_2, a))\]
- \[F = \setbuild{(p_1, p_2)}{p_1 \in F_1 {\color{red} \text{ and }} p_2 \in F_2}\]

Just update the accept states! (Note now \(F = F_1 \times F_2\))

Closure under intersection: Alternate proof

Theorem: the DFA-decidable languages are closed under intersection:

Alternatively, we can appeal to existing complement and union closure properties:
1. \(\setcomplement{A \cap B} = \setcomplement{A} \cup \setcomplement{B}\) by De Morgan’s Laws. (Boolean algebra equivalent: \(\neg(x \wedge y) \iff \neg x \vee \neg y\).
2. \(\setcomplement{A}\) and \(\setcomplement{B}\) are DFA-decidable by closure under complement.
3. \(\setcomplement{A} \cup \setcomplement{B}\) is DFA-decidable by closure under union, which equals \(\setcomplement{A \cap B}\) by step 1.
4. Thus, \(A \cap B\) is DFA-decidable by closure under complement.
Intuitively: just like any Boolean function can be built with \(\lor\) and \(\neg\), any set operation can be built from \(\cup\) and complement (or, equivalently, \(\cap\) and complement).
- Above, we showed that \(A \cap B = \setcomplement{(\setcomplement{A} \cup \setcomplement{B})}\).
- In other words, you can start with two DFA-decidable languages \(A,B\) and combine them using complement and union operations to get \(A \cap B\), which by closure under union and complement results in a DFA-decidable language.

Closure under multiple intersection and union

Under which of the following operations are DFA-decidable languages closed?

Finite union: \(\bigcup_{i=1}^n L_i\) for any \(n \in \mathbb{N}\)
Infinite union: \(\bigcup_{i=1}^\infty L_i\)
Finite intersection: \(\bigcap_{i=1}^n L_i\) for any \(n \in \mathbb{N}\)
Infinite intersection: \(\bigcap_{i=1}^\infty L_i\)

Closure under finite intersection and union can be proved easily by induction on \(n\).
- Assuming any union of \(n \in \mathbb{N}\) DFA-decidable languages is DFA-decidable, then any union of \(n+1\) also is: \[\bigcup_{i = 1}^{n + 1} L_i \fragment{= L_1 \cup \ldots \cup L_n \cup L_{n+1}} \fragment{= \underbrace{(L_1 \cup \ldots \cup L_n)}_{\bigcup_{i = 1}^{n} L_i \text{ is DFA decidable! (inductive hypothesis)}} \cup L_{n+1}}\]
What about infinite intersection and union?
- They can be used to construct any language, e.g., \(\setbuild{a^n b^n}{n \in \mathbb{N}} = \{\emptystring\} \cup \{ab\} \cup \{aabb\} \cup \{aaabbb\} \cup \dots\).

Closure in reverse

True or False: If \(A \cup B\) is DFA-decidable, then \(A\) and \(B\) are both DFA-decidable.

True
False

Closure is a “one-way property:”
- If \(A\) and \(B\) are DFA-decidable, then \(A \cup B\) is DFA-decidable, but the converse is not true.
  
  \[\begin{align*} A, B \text{ are DFA-decidable} \; &\Longrightarrow \; A \cup B \text{ is DFA-decidable} \\ &\cancel{\Longleftarrow} \end{align*}\]
For example, let \(A\) be a non-DFA-decidable language.
- Then \(B=\overline{A}\) is non-DFA-decidable as well.
- But \(A \cup \overline{A} = \Sigma^*\) and \(A \cap \overline{A} = \emptyset\) are both DFA-decidable.
Stated differently, the non-DFA-decidable languages are not closed under \(\cup\) or \(\cap\)…

Remaining operations for DFA closure

Complement: \(\overline{A} = \Sigma^* \setminus A = \setbuild{x \in \Sigma^*}{x \notin A}\)
Union: \(A \cup B = \setbuild{x \in \Sigma^*}{x \in A \text{ or } x \in B}\)
Intersection: \(A \cap B = \setbuild{x \in \Sigma^*}{x \in A \text{ and } x \in B}\)

Concatenation: \(A B = \setbuild{x_1 x_2}{x_1 \in A \text{ and } x_2 \in B}\)
Kleene Star: \(A^* = \setbuild{x_1 x_2 \ldots x_k}{k \geq 0 \text{ and } x_i \in A}\)

Let’s try to prove closure of DFA-decidability under concatenation:
- A string \(w\) is in \(L(D_1) \circ L(D_2)\) if it can be split into two parts \(w = x y\) such that
  \(x \in L(D_1)\) and \(y \in L(D_2)\). Idea: simulate \(D_1\) first (on \(x\)) and then \(D_2\) (on \(y\))!
- We feed \(x\) into \(D_1\), see if it ends in an accept state, then feed the rest \(y\) into \(D_2\) and see it also ends in an accept state. Problem?
- Where do we split \(w\) to form \(x\) and \(y\)???
- Can we just switch from \(D_1\) to \(D_2\) the moment \(D_1\) is about to reach an accept state?

DFA concatenation attempt

Let’s consider naïvely attempting concatenation for these two DFAs:

\(D_1\) \(D_2\)

If we immediately “jump” from even to q0 by redirecting the transition \(\texttt{even} \stackrel{1}{\to} \texttt{odd}\) instead to \(\texttt{even} \stackrel{1}{\to} \texttt{q0}\), then \(1111 = 111 \circ 1\) is rejected despite \(111 \in L(D_1)\) and \(1 \in L(D_2)\).

We don’t know when to transition to q0, and even if we did, we don’t know how to avoid doing it the very first time we encounter a state with a transition to q0.

The same “where and how to split?” difficulty is encountered for Kleene star.

Title

Closure of language classes

Roadmap for the next two chapters

Closure of DFA-decidable languages under complement

Complementing an NFA

Regex closure under union, concatenation, and star

DFA union

DFA union construction (product construction) example

DFA union construction (product construction)

Closure under intersection

Closure under intersection

Closure under intersection: Alternate proof

Closure under multiple intersection and union

Closure in reverse

Remaining operations for DFA closure

DFA concatenation attempt