Title

ECS 120 Theory of Computation
Closure of Language Classes and Algorithms on Automata
Julian Panetta
University of California, Davis

Review: DFA Union

  • To gain practice with designing constructions, let’s prove the following

    Theorem: the class of DFA-decidable languages is closed under union.

  • In other words, if \(L_1 = L(D_1)\) and \(L_2 = L(D_2)\), then we must prove the existence of a DFA \(D\) such that \(L_1 \cup L_2 = L(D)\).

    • Idea: simulate \(D_1\) and \(D_2\) in parallel, and accept if either \(D_1\) or \(D_2\) accepts.

      data/images/dfa/union/len_mod3_eq2/diagram.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/diagram.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)

      data/images/dfa/union/len_mod3_eq2/state_q0.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q0.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string:
      data/images/dfa/union/len_mod3_eq2/state_q1.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q1.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: a
      data/images/dfa/union/len_mod3_eq2/state_q2.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q2.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aa
      data/images/dfa/union/len_mod3_eq2/state_q0.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q3.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaa
      data/images/dfa/union/len_mod3_eq2/state_q1.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q4.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaa
      data/images/dfa/union/len_mod3_eq2/state_q2.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q0.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaa
      data/images/dfa/union/len_mod3_eq2/state_q0.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q1.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q1.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q2.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q2.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q3.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q0.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q4.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q1.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q0.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q2.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q1.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q0.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q2.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q1.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q3.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q2.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q4.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaaaaaaa
      data/images/dfa/union/len_mod3_eq2/state_q0.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/state_q0.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)
      Input string: aaaaaaaaaaaaaaa

    • What do we need to keep track of to execute this simulation?
      Which pair of states are the machines in?

DFA union construction (product construction) example

0 1 2 a a a 0 1 a 2 a a 3 a 4 a (0,0) (0,1) (0,2) (0,3) (0,4) (1,0) (1,1) (1,2) (1,3) (1,4) (2,0) (2,1) (2,2) (2,3) (2,4)

Formally:

  • \(Q = \fragment{\{0, 1, 2\} \times \{0, 1, 2, 3, 4\}}\)
  • \(s = (0, 0)\)
  • \(\delta((i, j), \string{a}) = (\fragment{i + 1 \mod 3}, \fragment{j + 1 \mod 5})\)
  • \(F = \setbuild{(i, j)}{\fragment{i = 2 \text{ or } (j = 1 \text{ or } j = 4)}}\)

DFA union construction (product construction)

  • Let \(D_1 = (Q_1, \Sigma, \delta_1, s_1, F_1)\) and \(D_2 = (Q_2, \Sigma, \delta_2, s_2, F_2)\) be two DFAs.

  • We construct a new DFA \(D = (Q, \Sigma, \delta, s, F)\) to decide \(L(D_1) \cup L(D_2)\)
    by simulating \(D_1\) and \(D_2\) in parallel:

    • \(Q = Q_1 \times Q_2 = \setbuild{(p_1, p_2)}{p_1 \in Q_1 \text{ and } p_2 \in Q_2}\)
    • \(s = \fragment{(s_1, s_2)}\)
    • Define \(\delta\) for all \((p_1, p_2) \in Q\) and \(a \in \Sigma\) as: \[\delta((p_1, p_2), a) = (\fragment{\delta_1(p_1, a)}, \fragment{\delta_2(p_2, a)})\]
    • Mark the accepting states: \[F = \setbuild{(p_1, p_2)}{\fragment{p_1 \in F_1 \text{ or } p_2 \in F_2}}\]
  • Then after reading a string \(w\) that puts \(D_1\) in state \(p_1\) and \(D_2\) in state \(p_2\),
    the new DFA \(D\) will be in state \((p_1, p_2)\).

    • \((p_1, p_2)\) is an accept state if and only if \(p_1 \in F_1\) or \(p_2 \in F_2\).
      So \(w\) will be accepted iff \(w \in L(D_1)\) or \(w \in L(D_2)\).
    • Thus \(L(D) = L(D_1) \cup L(D_2)\).

Closure Under Intersection

We already showed that the DFA-decidable languages are closed under union:
If \(L_1 = L(D_1)\) and \(L_2 = L(D_2)\), then we can construct \(D\) such that \(L(D) = L_1 \cup L_2\).

data/images/dfa/union/product-construction.svg

Formally:

  • Let \(D_1 = (Q_1, \Sigma, \delta_1, s_1, F_1)\) and
    \(\phantom{\text{Let }} D_2 = (Q_2, \Sigma, \delta_2, s_2, F_2)\)
  • Construct \(D = (Q, \Sigma, \delta, s, F)\):
    • \(Q = Q_1 \times Q_2 = \setbuild{(p_1, p_2)}{p_1 \in Q_1 \text{ and } p_2 \in Q_2}\)
    • \(s = (s_1, s_2)\)
    • For all \((p_1, p_2) \in Q\) and \(a \in \Sigma\): \[\delta((p_1, p_2), a) = (\delta_1(p_1, a), \delta_2(p_2, a))\]
    • \[F = \setbuild{(p_1, p_2)}{p_1 \in F_1 \text{ or } p_2 \in F_2}\]

What about intersection?

Closure Under Intersection

Theorem: the DFA-decidable languages are closed under intersection:
If \(L_1 = L(D_1)\) and \(L_2 = L(D_2)\), then we can construct \(D\) such that \(L(D) = L_1 \color{red} \cap \color{black} L_2\).

data/images/dfa/product-construction_intersection.svg

Formally:

  • Let \(D_1 = (Q_1, \Sigma, \delta_1, s_1, F_1)\) and
    \(\phantom{\text{Let }} D_2 = (Q_2, \Sigma, \delta_2, s_2, F_2)\)
  • Construct \(D = (Q, \Sigma, \delta, s, F)\):
    • \(Q = Q_1 \times Q_2 = \setbuild{(p_1, p_2)}{p_1 \in Q_1 \text{ and } p_2 \in Q_2}\)
    • \(s = (s_1, s_2)\)
    • For all \((p_1, p_2) \in Q\) and \(a \in \Sigma\): \[\delta((p_1, p_2), a) = (\delta_1(p_1, a), \delta_2(p_2, a))\]
    • \[F = \setbuild{(p_1, p_2)}{p_1 \in F_1 {\color{red} \text{ and }} p_2 \in F_2}\]

Just update the accept states!

Closure Under Intersection: Alternate Proof

Theorem: the DFA-decidable languages are closed under intersection:
If \(L_1 = L(D_1)\) and \(L_2 = L(D_2)\), then we can construct \(D\) such that \(L(D) = L_1 \color{red} \cap \color{black} L_2\).

  • Alternatively, we can appeal to existing complement and union closure properties:
    1. \(\setcomplement{L_1 \cap L_2} = \setcomplement{L_1} \cup \setcomplement{L_2}\) by De Morgan’s Laws. (Boolean algebra equivalent: \(\neg(x \wedge y) \iff \neg x \vee \neg y\)).
    2. \(\setcomplement{L_1}\) and \(\setcomplement{L_2}\) are DFA-decidable by closure under complement.
    3. \(\setcomplement{L_1} \cup \setcomplement{L_2}\) is DFA-decidable by closure under union, which equals \(\setcomplement{L_1 \cap L_2}\) by step 1.
    4. Thus, \(L_1 \cap L_2\) is DFA-decidable by closure under complement.
  • Intuitively: just like any Boolean function can be built with \(\lor\) and \(\neg\), any set operation can be built from \(\cup\) and complement (or, equivalently, \(\cap\) and complement).

Closure Under Multiple Intersection and Union

Under which of the following operations are DFA-decidable languages closed?

  • Finite union: \(\bigcup_{i=1}^n L_i\) for any \(n \in \mathbb{N}\)
  • Infinite union: \(\bigcup_{i=1}^\infty L_i\)
  • Finite intersection: \(\bigcap_{i=1}^n L_i\) for any \(n \in \mathbb{N}\)
  • Infinite intersection: \(\bigcap_{i=1}^\infty L_i\)
  • Closure under finite intersection and union can be proved easily by induction on \(n\).
    • Assuming any union of \(n \in \mathbb{N}\) DFA-decidable languages is DFA-decidable, then any union of \(n+1\) also is: \[\bigcup_{i = 1}^{n + 1} L_i \fragment{= L_1 \cup \ldots \cup L_n \cup L_{n+1}} \fragment{= \underbrace{(L_1 \cup \ldots \cup L_n)}_{\bigcup_{i = 1}^{n} L_i \text{ is DFA decidable! (inductive hypothesis)}} \cup L_{n+1}}\]
  • This does not extend to infinite intersection and union since those operations can be used to construct any language.
    • For example, \(\setbuild{a^n b^n}{n \in \mathbb{N}} = \bigcup_{n=0}^\infty \{a^n b^n\} = \{\emptystring\} \cup \{ab\} \cup \{aabb\} \cup \{aaabbb\} \cup \dots\).

Potential Pitfall

  • Closure is a “one-way property:”
    • If \(L_1\) and \(L_2\) are DFA-decidable, then \(L_1 \cup L_2\) is DFA-decidable.

      \[ L_1, L_2 \text{ DFA-decidable} \; \Longrightarrow \; L_1 \cup L_2 \text{ DFA-decidable} \]

    • But if \(L_1 \cup L_2\) is DFA-decidable, then \(L_1\) and \(L_2\) are not necessarily DFA-decidable.

      \[ L_1, L_2 \text{ DFA-decidable} \cancel{\Longleftarrow} L_1 \cup L_2 \text{ DFA-decidable} \]

  • For example, let \(A\) be a non-DFA-decidable language.
    • Then \(\overline{A}\) is non-DFA-decidable as well.
    • But \(A \cup \overline{A} = \Sigma^*\) and \(A \cap \overline{A} = \emptyset\) are both DFA-decidable.
  • Stated differently, the non-DFA-decidable languages are not closed under \(\cup\) or \(\cap\)

Remaining Operations

Let’s survey our progress on proving closure properties for DFA-decidable languages.

  • Complement: \(\overline{L} = \Sigma^* \setminus L = \setbuild{w \in \Sigma^*}{w \notin L}\)
  • Union: \(L_1 \cup L_2 = \setbuild{w \in \Sigma^*}{w \in L_1 \text{ or } w \in L_2}\)
  • Intersection: \(L_1 \cap L_2 = \setbuild{w \in \Sigma^*}{w \in L_1 \text{ and } w \in L_2}\)
  • Concatenation: \(L_1 L_2 = \setbuild{w_1 w_2}{w_1 \in L_1 \text{ and } w_2 \in L_2}\)
  • Kleene Star: \(L^* = \setbuild{w_1 w_2 \ldots w_k}{k \geq 0 \text{ and } w_i \in L}\)
  • Let’s try to prove closure under concatenation:
    • A string \(w\) is in \(L(D_1) \circ L(D_2)\) if it can be split into two parts \(w = w_1 w_2\) such that
      \(w_1 \in L(D_1)\) and \(w_2 \in L(D_2)\). Idea: simulate \(D_1\) first (on \(w_1\)) and then \(D_2\) (on \(w_2\))!
    • We feed \(w_1\) into \(D_1\) and \(w_2\) into \(D_2\) and see if they both end up in an accept state. Problem?
    • Where do we split \(w\) to form \(w_1\) and \(w_2\)???
    • Can we just switch from \(D_1\) to \(D_2\) the moment \(D_1\) reaches an accept state?

DFA Concatenation Attempt

Let’s consider the concatenation of
\(L_1 = \setbuild{w \in \{0, 1\}^*}{ \#_1(w) \equiv 1 \mod 2}\), and \(L_2 = \setbuild{w \in \{0, 1\}^*}{ \#_1(w) \equiv 1 \mod 3}\)

data/images/dfa/parity_odd.svg data/images/dfa/one_mod3.svg

If we immediately “jump” from odd to \(q_0\) by redirecting the transition \(\texttt{even} \stackrel{1}{\to} \texttt{odd}\) into \(\texttt{even} \stackrel{1}{\to} \texttt{q0}\), then the string \(1111 = 111 \circ 1\) is rejected despite \(111 \in L_1\) and \(1 \in L_2\).

The same “where to split?” difficulty is encountered for Kleene star.

NFAs to the Rescue

  • The more convenient and flexible NFA model makes it simpler to construct a machine deciding the concatenation and Kleene star of languages.

  • Once we demonstrate how to convert an NFA to a DFA, this will prove that DFA-decidable languages are closed under concatenation and Kleene star.

  • First: how can we simplify the union construction with NFAs?

    data/images/dfa/union/len_mod3_eq2/diagram.svg\(L(D_1) = \setbuild{a^n}{n \equiv 2 \mod 3}\) data/images/dfa/union/len_mod5/diagram.svg\(L(D_2) = \setbuild{a^n}{n \equiv 1 \mod 5 \; \text{ or } \; n \equiv 4 \mod 5}\)

    Just create a new start state and add \(\emptystring\)-transitions to the start states of \(D_1\) and \(D_2\).

NFAs to the Rescue

  • The more convenient and flexible NFA model makes it simpler to construct a machine deciding the concatenation and Kleene star of languages.

  • Once we demonstrate how to convert an NFA to a DFA, this will prove that DFA-decidable languages are closed under concatenation and Kleene star.

  • First: how can we simplify the union construction with NFAs?

    data/images/nfa/union/nfa_union.svg

    Just create a new start state and add \(\emptystring\)-transitions to the start states of \(D_1\) and \(D_2\).

NFAs to the rescue: Concatenation example

Let’s consider the concatenation of
\(L_1 = \setbuild{w \in \{0, 1\}^*}{ \#_1(w) \equiv 1 \mod 2}\), and
\(L_2 = \setbuild{w \in \{0, 1\}^*}{ \#_1(w) \equiv 1 \mod 3}\)

q0 q0 7199cb39-93c7-4201-ac0e-4df6530211cc 7199cb39-93c7-4201-ac0e-4df6530211cc->q0 q0->q0 0 q1 q1 q0->q1 1 q1->q1 0 q2 q2 q1->q2 1 q2->q0 1 q2->q2 0 e373b78d-04ad-4784-82ed-e2431916b441 even even e373b78d-04ad-4784-82ed-e2431916b441->even even->even 0 odd odd even->odd 1 odd ->even 1 odd ->odd 0 odd ->even

NFAs to the rescue: Another concatenation example

Let’s consider the concatenation of
\(L(N_1) = \setbuild{w \in \{0, 1\}^*}{ \texttt{int}(w, 2) \not \equiv 2 \mod 3 }\), and
\(L(N_2) = \setbuild{w \in \{0, 1\}^*}{ \texttt{int}(w, 2) \equiv 2 \mod 3}\)

Here \(\texttt{int}(w, 2)\) is the integer value of \(w\) in base 2.

d26f7849-df09-4d93-91c3-925ca4118297 q0 q0 d26f7849-df09-4d93-91c3-925ca4118297->q0 q0->q0 0 q1 q1 q0->q1 1 q1->q0 1 q2 q2 q1->q2 0 q2->q1 0 q2->q2 1 ba978f6c-6209-416b-8526-f6d2876fc697 r0 r0 ba978f6c-6209-416b-8526-f6d2876fc697->r0 r0->r0 0 r1 r1 r0->r1 1 r1->r0 1 r2 r2 r1->r2 0 r2->r1 0 r2->r2 1 d26f7849-df09-4d93-91c3-925ca4118297 q0 d26f7849-df09-4d93-91c3-925ca4118297->q0 q0->q0 0 q1 q0->q1 1 q1->q0 1 q2 q2 q1->q2 0 q2->q1 0 q2->q2 1 r0 r0 ba978f6c-6209-416b-8526-f6d2876fc697 ba978f6c-6209-416b-8526-f6d2876fc697->r0 r0->r0 0 r1 r1 r0->r1 1 r1->r0 1 r2 r2 r1->r2 0 r2->r1 0 r2->r2 1 odd ->even odd ->even

NFAs to the rescue: Kleene star example

Consider the star of: \(L(N_1) = \setbuild{w \in \{\string{a, b}\}^*}{ w \text{ has an odd number of } \string{b} \text{'s followed by an } \string{a}}\)

data/images/nfa/odd_b_followed_a.svg
data/images/nfa/odd_b_followed_a_star_bad.svg

This changes the behavior of the original NFA \(N_1\)! … accepts \(\string{bb}\) even though \(\string{bb} \notin L(N_1)^*\)

NFAs to the rescue: Kleene star example

Consider the star of: \(L(N_1) = \setbuild{w \in \{\string{a, b}\}^*}{ w \text{ has an odd number of } \string{b} \text{'s followed by an } \string{a}}\)

data/images/nfa/odd_b_followed_a.svg
data/images/nfa/odd_b_followed_a_star.svg