Undecidability of the Halting Problem

So far we have assumed that the halting problem is undecidable: \[ \probHALT = \setbuild{\encoding{M, w}}{M \text{ is a Turing machine that halts on } w} \] and used this to prove that other problems are undecidable using Turing reductions.

In the final lecture, we will prove that the halting problem is undecidable using Turing’s original proof from 1936.
Turing’s proof was based on the diagonalization technique originally developed by Cantor to prove the existence of different sizes of infinity.
- There are an infinite number of reals in \(\R\) and natural numbers in \(\N\)
- But actually the infinite size of \(\R\) is larger than the infinite size of \(\N\) in a rigorous sense.
We will warm up to this by studying the sizes of sets.

Comparing the Sizes of Sets

How can we compare the sizes of two sets?
For finite sets, we can just count the number of elements in each set and compare.
But this breaks down for infinite sets!
To compare \(\N\) and \(\R\), we might be tempted to claim \(\N\) is obviously smaller since \(\N \subsetneq \R\).
But checking if \(A \subsetneq B\) is a bad way to determine whether \(|A| < |B|\).
- Clearly \(|\{1, 2\}| < |\{3, 4, 5\}|\), but \(\{1, 2\} \not \subseteq \{3, 4, 5\}\).
- Also, \(\N \subsetneq \Z\), but actually we will see that \(\N\) and \(\Z\) are the “same size.”
Cantor’s idea was to compare sets by finding correspondences between them.
- Two finite sets \(A\) and \(B\) are the same size if and only if we can find a bijection between them.
  
  Recall: a bijection \(f: A \to B\)
  is both one-to-one (injective) and onto (surjective).
- Furthermore, \(|A| < |B|\) if and only if there is no onto function \(f: A \to B\).
  (Equivalently, there is no injective function \(f: B \to A\).)

No onto function exists, so
\(|\{1, 2\}| < |\{3, 4, 5\}|\)

Comparing the Sizes of Sets

We can use this approach of comparing sizes by establishing correspondences to define what it means for one set to be smaller than another:

For any two sets \(A\) and \(B\), we say that \(|A| \ge |B|\) if and only if there exists an onto function \(f: A \to B\).

In the previous example, we can exhibit an onto function from \(\{3, 4, 5\}\) to \(\{1, 2\}\) to conclude \(|\{3, 4, 5\}| \ge |\{1, 2\}|\). \[ f(3) = 1, \quad f(4) = 2, \quad f(5) = 2. \]
But now we can apply the definition to compare infinite sets!

Comparing \(|\N|\) and \(|\Z|\)

Recall that \(\N\) denotes the set of nonnegative integers, and \(\Z\) denotes all integers. \[ \N = \{0, 1, 2, \ldots\}, \quad \quad \Z = \{\ldots, -2, -1, 0, 1, 2, \ldots\} \]
To compare their sizes, we look for onto functions between them.
It is easy to find onto functions from \(\Z\) to the “apparently smaller” \(\N\). So \(|\Z| \ge |\N|\).
- What’s a good choice for \(f\)? \(f(n) = \abs{n}\)
- Many other choices are possible, e.g. \(f(n) = \max(n, 0)\).
- In fact, whenever \(A \subseteq B\) we can construct an onto function \(f: B \to A\): \[ f(x) = \begin{cases} x & \text{ if } x \in A \\ a_0 & \text{otherwise.} \end{cases}, \quad \quad \text{where } a_0 \in A \text{ is arbitrary.} \]
- \(f(n) = \max(n, 0)\) is an example of this general construction for \(B = \Z\) and \(A = \N\), with \(a_0 = 0\).

Can we find an onto function from \(\N\) to \(\Z\)?

Comparing \(|\N|\) and \(|\Z|\)

Recall that \(\N\) denotes the set of nonnegative integers, and \(\Z\) denotes all integers. \[ \N = \{0, 1, 2, \ldots\}, \quad \quad \Z = \{\ldots, -2, -1, 0, 1, 2, \ldots\} \]
To compare their sizes, we look for onto functions between them.
It is easy to find onto functions from \(\Z\) to the “apparently smaller” \(\N\). So \(|\Z| \ge |\N|\).

We can find an onto function from \(\N\) to \(\Z\) by “zig-zagging” through the integers.

\[ \begin{aligned} \\ &0 \quad 1 \quad 2 \quad 3 \quad 4 \quad 5 \quad 6 \quad 7 \quad 8 \quad \cdots \\ \\ \cdots \quad -4 \quad -3 \quad -2 \quad -1 \quad &0 \quad 1 \quad 2 \quad 3 \quad 4 \quad \cdots\\ \end{aligned} \]

\[ \fragment{ f(n) = \begin{cases} -\frac{n}{2} & \text{ if } n \text{ is even.} \\ \left \lceil \frac{n}{2} \right \rceil & \text{ if } n \text{ is odd} \end{cases} } \]
Thus also \(|\N| \ge |\Z|\), and so \(|\N| = |\Z|\).

Comparing \(|\N|\) and \(|\Q|\)

Let’s now consider the rational numbers \(\Q\). \[ \Q = \setbuild{\frac{n}{d}}{n \in \Z, d \in \N^+ } \]

It looks like there are way more of these!
We’ll start out with just the positive rationals \(\Q^+\).
Clearly \(|\Q^+| \ge |\N^+|\) since \(\N^+ \subsetneq \Q^+\), and thus \(|\Q^+| \ge |\N|\).
True or false: \(|\N^+| \ge |\N|\)?
- True
- False

Comparing \(|\N|\) and \(|\Q|\)

Let’s now consider the rational numbers \(\Q\). \[ \Q = \setbuild{\frac{n}{d}}{n \in \Z, d \in \N^+ } \]

It looks like there are way more of these!
We’ll start out with just the positive rationals \(\Q^+\).
Clearly \(|\Q^+| \ge |\N^+|\) since \(\N^+ \subsetneq \Q^+\), and thus \(|\Q^+| \ge |\N|\).

But what about finding an onto \(f : \N \to \Q^+\)?
Finding a function from \(\N\) to another set is just enumerating the elements of that set. \[ r_0 = f(0), \quad \fragment{r_1 = f(1), \quad} \fragment{r_2 = f(2), \quad} \fragment{r_3 = f(3), \quad \cdots} \]
How can we enumerate \(\Q^+\)?
- The following won’t work: \[ \frac{1}{1}, \; \fragment{\frac{1}{2},} \; \fragment{\frac{1}{3},} \; \fragment{\frac{1}{4}, \; \cdots,} \; \fragment{\frac{2}{1},} \; \fragment{\frac{2}{2},} \; \fragment{\frac{2}{3}, \; \frac{2}{4}, \; \cdots,} \; \fragment{\frac{3}{1}, \; \frac{3}{2}, \; \frac{3}{3}, \; \cdots} \hspace{20em} \]
  
  Larger numerators aren’t reached at any finite step! Numerator and denominator must change together!

Comparing \(|\N|\) and \(|\Q|\)

Let’s now consider the rational numbers \(\Q\). \[ \Q = \setbuild{\frac{n}{d}}{n \in \Z, d \in \N^+ } \]

It looks like there are way more of these!
We’ll start out with just the positive rationals \(\Q^+\).
Clearly \(|\Q^+| \ge |\N^+|\) since \(\N^+ \subsetneq \Q^+\), and thus \(|\Q^+| \ge |\N|\).

But what about finding an onto \(f : \N \to \Q^+\)?
Finding a function from \(\N\) to another set is just enumerating the elements of that set. \[ r_0 = f(0), \quad r_1 = f(1), \quad r_2 = f(2), \quad r_3 = f(3), \quad \cdots \]
How can we enumerate \(\Q^+\)?
- We can list them in order of \(n + d\)!
- This way every positive rational
  is eventually reached.

	1	2	3	4
1	1/1	2/1	3/1	4/1
2	1/2	2/2	3/2	4/2
3	1/3	2/3	3/3	4/3
4	1/4	2/4	3/4	4/4

Comparing \(|\N|\) and \(|\Q|\)

We’ve now shown the existence of an onto function \(f: \N \to \Q^+\).
Therefore \(|\N| \ge |\Q^+|\) (and thus \(|\N| = |\Q^+|\)).
What about \(|\Q|\)?
- We can use \(f\) to construct a new onto function \(g: \N \to \Q\). \[ \begin{aligned} g(0) &= 0, \\ g(1) &= \phantom{-}f(0), \\ g(2) &= -f(0), \\ g(3) &= \phantom{-}f(1), \\ g(4) &= -f(1), \\ g(5) &= \phantom{-}f(2), \\ g(6) &= -f(2), \\ \vdots & \\ \end{aligned} \quad \quad \fragment{ g(n) = \begin{cases} 0 & \text{ if } n = 0 \\ f\left(\frac{n+1}{2}\right) & \text{ if } n \text{ is odd} \\ -f\left(\frac{n}{2}\right) & \text{ if } n \text{ is even} \end{cases}} \]
Since \(f\) maps onto every positive rational, \(g\) maps onto every rational number.
Hence \(|\N| \ge |\Q|\) (and thus \(|\N| = |\Q|\)).

Comparing \(|\N|\) and \(|\binary^*|\)

We can also compare sizes of sets that aren’t just holding numbers!
For example: let’s compare \(\N\) with the set of all binary strings \(\binary^*\).
We do this by enumerating the binary strings.
We’ve already seen this enumeration! It’s the length-lexicographic ordering: \[ \emptystring, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, 100, 101, 110, 111, \ldots \]
We conclude that \(|\N| \ge |\binary^*|\).
The enumeration above actually represents a bijection so actually \(|\N| = |\binary^*|\).

Did our earlier enumeration of \(\Q\) correspond to a bijection?

Comparing \(|\R|\) and the (Open) Unit Interval \((0, 1)\)

Now let’s consider some huge sets:
- The set of all real numbers \(\R\).
- The open unit interval \((0, 1) \subsetneq \R\).
Clearly \(|\R| \ge |(0, 1)|\) since \((0, 1) \subsetneq \R\).
Surprisingly, there are the same number of real numbers in \(\R\) and in \((0, 1)\)!
We first show the easier fact that \(|(0, 1)| \ge |\R^+|\).
- What function can we use? One example: \(g(x) = \frac{1}{x} - 1\)
- Note that for any \(y \in \R^+\), we can solve the following for \(x \in (0, 1)\): \[ g(x) = y \quad \iff \quad x = \frac{1}{y + 1}. \]

Comparing \(|\R|\) and the (Open) Unit Interval \((0, 1)\)

Now let’s consider some huge sets:
- The set of all real numbers \(\R\).
- The open unit interval \((0, 1) \subsetneq \R\).
Clearly \(|\R| \ge |(0, 1)|\) since \((0, 1) \subsetneq \R\).
Surprisingly, there are the same number of real numbers in \(\R\) and in \((0, 1)\)!
We first show the easier fact that \(|(0, 1)| \ge |\R^+|\).

Second, we show that \(|\R^+| \ge |\R|\):
- What function can we use? One example: \(f(x) = \log(x)\)
- Note that for any \(y \in \R\), we can solve the following for \(x \in \R^+\): \[ f(x) = y \quad \iff \quad x = e^y. \]

We conclude that \(|(0, 1)| \ge |\R^+| \ge |\R|\) and so \(|(0, 1)| = |\R|\).

Title

Undecidability of the Halting Problem

Comparing the Sizes of Sets

Comparing the Sizes of Sets

Comparing \(|\N|\) and \(|\Z|\)

Comparing \(|\N|\) and \(|\Z|\)

Comparing \(|\N|\) and \(|\Q|\)

Comparing \(|\N|\) and \(|\Q|\)

Comparing \(|\N|\) and \(|\Q|\)

Comparing \(|\N|\) and \(|\Q|\)

Comparing \(|\N|\) and \(|\binary^*|\)

Comparing \(|\R|\) and the (Open) Unit Interval \((0, 1)\)

Comparing \(|\R|\) and the (Open) Unit Interval \((0, 1)\)