Title

ECS 120 Theory of Computation
Week 1 Discussion: Discrete Math Review
Dave Doty and Julian Panetta
University of California, Davis

Mathematical Notation and Terminology

Sets and Sequences

Sets

  • A set is a collection of objects called elements or members.
    • Contains no duplicates.
    • Order doesn’t matter.
    • Can be empty, finite or infinite.
  • Corresponding programming data types:
    • Python: set,
    • Java: java.util.Set,
    • C++: std::set, std::unordered_set

Note: mathematical sets (data types like std::set) are “immutable”

  • they cannot grow, shrink, or change in any way.
  • To “add to a set”, define a new set: \(S_0 = \{\text{stuff}\}\) and let \(S_n = S_{n-1} \cup \{\text{more stuff}\}\).

Set Notation

  • If the set is finite (and small), we can list its elements explicitly:
    • \(S = \{2, 5, 7\}\) contains three elements: 2, 5, and 7
    • \(|S| = 3\) is the cardinality of \(S\): the number of elements it contains
    • \(7 \in S\) denotes that 7 is an element of \(S\).
    • \(8 \notin S\) denotes that 8 is not an element of \(S\).
  • \(A \subseteq B\) denotes that \(A\) is a subset of \(B\)
    • For every \(x \in A\), it is also the case that \(x \in B\)
    • \(\{2, 7\} \subseteq \{2, 5, 7\}\) and \(\{2, 7\} \subseteq \{2, 7\}\)
  • \(A = B\) means that \(A\) and \(B\) contain the same elements (\(A \subseteq B\) and \(B \subseteq A\))
  • \(A \subsetneq B\) denotes that \(A\) is a proper subset of \(B\) (\(A \subseteq B\) but \(A \neq B\))
    • \(\{2, 7\} \subsetneq \{2, 5, 7\}\)

    (Some references write this as \(A \subset B\))

  • \(\emptyset = \{\}\) is the empty set: \(|\emptyset| = 0\), and \(x \notin \emptyset\) for all \(x\)
  • \(\{7\}\) is a singleton (one-element set), and \(\{2, 5\}\) is an unordered pair.

Common Sets with Standard Names

  • \(\N = \{ 0, 1, 2, 3, \ldots \}\) is the set of natural numbers (nonnegative integers).
    • some references instead define \(\N = \{ 1, 2, 3, \ldots \}\)
    • we call this \(\N^+ = \N \setminus \{ 0 \}\).
  • \(\Z = \{ \ldots, –2, –1, 0, 1, 2, \ldots \}\) is the set of integers.
  • \(\R\) is the set of real numbers
  • \(\Q = \setbuild{\frac{n}{d}}{n \in \Z, d \in \N^+ }\) is the set of rational numbers .
  • This definition of \(\Q\) is an example of set-builder notation.
    • Analogous to set comprehensions in Python
    • For example, {n/d for n in [-1,0,1] for d in [1,2,3]}
      creates the set {–1.0, –0.5, –0.333, 0.0, 0.333, 0.5, 1.0}.

Operations on Sets

  • The union of two sets \(A\) and \(B\):
    • \(A \cup B = \setbuild{x}{x \in A \text{ or } x \in B}\)
    • \(\{2, 5\} \cup \{5, 7\} = \{2, 5, 7\}\)
  • The intersection of two sets \(A\) and \(B\):
    • \(A \cap B = \setbuild{x}{x \in A \text{ and } x \in B}\)
    • \(\{2, 5\} \cap \{5, 7\} = \{5\}\)
  • The difference of two sets \(A\) and \(B\):
    • \(A \setminus B = A - B = \setbuild{x}{x \in A \text{ and } x \notin B}\)
    • \(\{2, 5\} \setminus \{5, 7\} = \{2\}\)
  • The complement of a set \(A\)
    (with respect to a universal set \(U\)):
    • \(\overline{A} = \setbuild{x \in U}{x \notin A} = U \setminus A\)
    • \(\overline{\{2, 5\}} = \{0, 1, 3, 4, 6, 7, 8, 9, \ldots\}\)
data/images/discussion1/venn_diagram.svg
Venn Diagram of sets \(A\), \(B\) within the universal set \(U = \N\).

Sets of Sets

  • Sets can contain other sets!

  • We will soon encounter classes of languages (sets of sets of strings).

  • The power set of a set \(S\) is the set of all subsets of \(S\).

    • \(\powerset(S) = \setbuild{A}{A \subseteq S}\)
    • \(\powerset(\{2, 5\}) = \{\emptyset, \{2\}, \{5\}, \{2, 5\}\}\)

  • What is the cardinality of \(\powerset(S)\)?

    • \(2 |S|\)
    • \(2^{|S|}\)
    • \(|S|^2\)
    • Not enough information

Sequences and Tuples

  • A sequence is an ordered list of elements.
    • Duplicate elements are allowed (unlike sets).
    • Order matters (unlike sets).
    • Can be finite or infinite.
  • Typically denoted with parentheses: \((2, 5, 7) \ne (5, 2, 7) \ne (5, 2, 7, 2)\).
  • A finite sequence is also called a tuple.
  • A tuple with \(k\) elements is an \(k\)-tuple.
    • ordered pair: 2-tuple
    • ordered triple: 3-tuple

Corresponding programming data types:

  • Python: list/tuple,
  • Java: java.util.List,
  • C++: std::vector
  • Arrays in any language.

Cartesian Product

  • The Cartesian product of two sets \(A\) and \(B\):
    • \(A \times B = \setbuild{(a, b)}{a \in A, b \in B}\)
    • \(\{2, 5\} \times \{5, 7\} = \{(2, 5), (2, 7), (5, 5), (5, 7)\}\)
    • The Python equivalent is {(a, b) for a in A for b in B}
      or set(itertools.product(A, B))
  • When \(A\) is an ordinary set (not interpreted as a language), \(A^2 = A \times A\)
  • More than two sets can be Cartesian-producted:
    • \(A \times B \times C = \setbuild{(a, b, c)}{a \in A, b \in B, c \in C}\) is a set of 3-tuples
    • \(A \times B \times C \times D\) is a set of 4-tuples, etc.

What is the cardinality of \(A^2\)?

  • \(2 |A|\)
  • \(2^{|A|}\)
  • \(|A|^2\)
  • Not enough information

Functions and Relations

Functions

  • Function: an input/output relationship (mapping)
  • \(f(a) = b\) expresses that given input \(a\), \(f\) outputs \(b\)
  • Differences between programming functions and mathematical functions:
    • A function in a program is an algorithm with steps indicating how to compute the output from the input.
    • A mathematical function is just the abstract input/output relationship itself;
      in fact there may be no algorithm computing the function!
    • Mathematical functions have no side effects (and no state).
  • The set of all possible inputs is called the domain \(D\) of the function.
  • The function outputs elements from a set called the range \(R\) of the function.
  • We indicate these facts with the notation \(f: D \to R\).
    • This is like a function signature in a programming language.
    • The C function signature “bool f(int a);” corresponds to a function \(f: \Z \to \{0, 1\}\).

Example Function Definitions

  • abs: take the absolute value of an integer

    • Type signature? \(\texttt{abs}: \Z \to \Z\) or \(\texttt{abs}: \Z \to \N\)

    • Definition: \[ \text{for all } x \in \Z, \quad \texttt{abs}(x) = \begin{cases} x & \text{if } x \ge 0 \\ -x & \text{otherwise} \end{cases} \]

    • Corresponding C code:

      int abs(int x) { return x >= 0 ? x : -x; }
  • add: add two integers together

    • Type signature? \(\texttt{add}: \Z \times \Z \to \Z\)

    • Definition: \[ \text{for all } x, y \in \Z, \quad \texttt{add}(x, y) = x + y \]

    • Corresponding C code:

      int add(int x, int y) { return x + y; }

Rather than a rule or formula, a function can be defined by a (potentially infinite) table of inputs and outputs.

Example: \[ f: \{0, 1, 2, 3, 4\} \to \{0, 1, 2, 3, 4\} \]

defined by:

\(n\) \(f(n)\)
0 1
1 2
2 3
3 4
4 0

(incrementing modulo 5)

One-to-one and Onto Functions

  • A function \(f: D \to R\) is one-to-one (or injective)
    if every element in \(D\) maps to a unique element in \(R\).
    • \(f(a) = f(b)\) implies \(a = b\).
  • A function \(f: D \to R\) is onto (or surjective)
    if every element in \(R\) is the image of some element in \(D\).
    • For every \(r \in R\), there exists \(d \in D\) such that \(f(d) = r\).
  • A function that is both one-to-one and onto
    is called bijective.
D R f 1 - 1, not onto: no two arrows “collide” onto, not 1 - 1: all elements of R are “covered” 1 - 1 and onto:

Predicate: a Function with Boolean Range

  • Example: \(\text{even}: \Z \to \{ \text{true}, \text{false} \}\) defined by \[ \text{even}(n) = \begin{cases} \text{true} & \text{if } n \text{ is even} \\ \text{false} & \text{otherwise} \end{cases} \]

  • Any predicate is identified by the subset of inputs that make it true
    (e.g., even is identified by the set of even integers).

  • Relation: a predicate with domain \(A^2 = A \times A\);
    equivalently, a subset of \(A^2\)

    • Example: the “less than” relation on \(\Z\) is the set of pairs \((a,b) \in \Z^2\) such that \(a < b\).

Equivalence Relations

  • Equivalence Relation: a relation \(\equivrel \subseteq A^2\) that is reflexive, symmetric, and transitive.
    • Reflexive: \(a \equivrel a\) for all \(a \in A\)
    • Symmetric: for every \(a, b \in A\), \(a \equivrel b\) implies \(b \equiv a\)
    • Transitive: for every \(a, b, c \in A\), \(a \equivrel b\) and \(b \equiv c\) implies \(a \equiv c\)
  • Examples:
    • The “equals” relation on \(\Z\) is an equivalence relation.
    • For \(x, y \in \N\), \(x \equivrel y\) if \(x\) and \(y\) have the same quotient when divided by 3. \((6 \equiv 7 \equiv 8, 9 \equiv 10 \equiv 11, \ldots)\)
    • For \(x, y \in \N\), \(x \equivrel y\) if \(x\) and \(y\) have the same remainder when divided by 3. \((1 \equiv 4 \equiv 7 ..., 2 \equiv 5 \equiv 8 \ldots, \ldots)\)
  • For \(x \in A\), write \([x] = \setbuild{y \in A}{x \equivrel y}\) to denote equivalence class of \(x\).
    • For the quotient equivalence relation, \([0] = \{0,1, 2\}\), \([3] = \{3, 4, 5\}\), etc.
    • For the remainder equivalence relation, \([0] = \{0, 3, 6, 9, \ldots\}\), \([1] = \{1, 4, 7, 10, \ldots\}\), \([2] = \{2, 5, 8, 11, \ldots\}\).

Graphs and Trees

Graphs

a c d b G = ( V , E ) nodes / vertices V = {a, b, c, d} edges E = { { a,b }, { a,d }, { b,d }, { c,d } } undirected graph degree of node = # of edges connected to it degree(d) = 3 w z y x edges E = { ( w,x ) , ( x,w ) , ( w,y ) , ( y,x ) , ( y,z ) } outdegree of node = # of edges leaving it directed graph indegree of node = # of edges entering it outdegree(y) = 2 indegree(y) = 1

Paths in Graphs

• path = sequence of nodes connected by edges (c, d, b, d, a) • simple path = path with no repeated nodes (c, d, b, a) • graph is connected if all pairs of nodes have a path between them • directed graph is strongly connected if all pairs of nodes have a directed path between them in each direction • cycle = path starting and ending with same node (c, d, a, d, c) • simple cycle = cycle repeating only first/last node (a, d, b, a) a c d b connected 1 4 3 2 disconnected 5 connected, but not strongly connected 1 4 3 2 strongly connected 1 4 3 2

Trees

• tree = connected graph with no cycles • rooted tree = tree with one special node designated as the root • leaf = non - root node of degree 1 a c d b r e a f c d h g b r d b c f e a

Boolean Logic

Boolean Logic

  • Two-valued logic: \(\{0, 1\}\) or \(\{\texttt{false}, \texttt{true}\}\), or \(\{\texttt{no}, \texttt{yes}\}\)
  • Boolean operations are conveniently defined via truth tables.

Negation/NOT

\[ \begin{array}{c|c} x & \neg x \\ \hline 0 & 1 \\ 1 & 0 \end{array} \]

Conjunction/AND

\[ \begin{array}{cc|c} x & y & x \land y \\ \hline 0 & 0 & 0 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \\ 1 & 1 & 1 \end{array} \]

Disjunction/OR

\[ \begin{array}{cc|c} x & y & x \lor y \\ \hline 0 & 0 & 0 \\ 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 1 \end{array} \]

More Boolean Operations

Exclusive or/XOR \[ \begin{array}{cc|c} x & y & x \oplus y \\ \hline 0 & 0 & 0 \\ 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{array} \]

Implication

\[ \begin{array}{cc|c} x & y & x \implies y \\ \hline 0 & 0 & 1 \\ 0 & 1 & 1 \\ 1 & 0 & 0 \\ 1 & 1 & 1 \end{array} \]

(“If x then y” or “x implies y”)

Equivalent to \(\neg x \lor y\).

Equality/IFF

\[ \begin{array}{cc|c} x & y & x \iff y \\ \hline 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \\ 1 & 1 & 1 \end{array} \]

“x if and only if y”

Equivalent to \((x \land y) \lor (\neg x \land \neg y)\).

Pigeonhole Principle

Pigeonhole Principle

Pigeonhole principle : If we put n objects (“pigeons”) a 1 , a 2 , ..., a n into fewer than n boxes, then there is a box with at least two objects. box 1 box 2 box 3 a 1 a 2 a 3 a 4 a 2 a 4 a 1 a 3 10 pigeons but only 9 pigeonholes

Example Applications

  • Pick \(5\) cards from a standard deck; at least two must be of the same suit (♣♦♥♠)
    • Pigeons: cards,
    • Boxes: suits
  • If you pick 5 distinct numbers from \(\{1, 2, 3, 4, 5, 6, 7, 8\}\), at least two must sum to 9
    • Pigeons: number from \(\{1, 2, 3, 4, 5, 6, 7, 8\}\)
    • Boxes: pairs that add to 9: \(\{(1, 8), (2, 7), (3, 6), (4, 5)\}\)
  • Every hash function from a set of size \(n\) to a set of size \(m\) must have a collision if \(n > m\)
    • Pigeons: elements in the domain,
    • Boxes: elements in the range
    • Similarly: no compression function can compress every file!

Combinatorics

Product Rule

• If there are x ways to pick one item and y ways to pick a second item, there are x∙y ways to pick both. • Example : If I have 3 shirts and 2 pairs of pants, how many outfits can I make? • A = {set of pants}, B = {set of shirts}, then {set of outfits} = A × B; recall

Product Rule Extends to \(> 2\) Items

• Example : 2 pairs of pants, 3 shirts, 4 pairs of shoes… How many outfits? 2∙3∙4 = 24 • Example : How many functions are there f :{1,…, n } → {1,…, k }? k ways to pick f (1) ∙ k ways to pick f (2) … ∙ k ways to pick f ( n ) = k n ways to pick all n values f (1), f (2), …, f ( n ) {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 {1,2,3} → {1,2,3,4} 1 3 2 1 3 2 4 f (1) f (2)

Example: Counting Strings of a Fixed Length

  • How many strings of length \(n\) can be formed from the alphabet \(\{0, 1\}\)?
    • Length 0: 1 string (\(\emptystring\))
    • Length 1: 2 strings (\(0\), \(1\))
    • Length 2: 4 strings (\(00\), \(01\), \(10\), \(11\))
    • Length 3: 8 strings (\(000\), \(001\), \(010\), \(011\), \(100\), \(101\), \(110\), \(111\))
    • Length \(n\): \(2^n\) strings

Three Different Counting Arguments

2 ∙ 2 ∙ … ∙ 2 ways to choose 1 st bit ways to choose 2 nd bit ways to choose n th bit direct counting induction base case : 1 string of length 0 (empty string) inductive case : The inductive hypothesis says there are 2 n strings of length n . Each length - n string x is a prefix of exactly 2 length - ( n +1) strings x 0 and x 1 , so there are 2∙2 n = 2 n +1 strings of length n +1. data structures length - n strings correspond to leaves of a complete binary tree of depth n 0 1 00 01 10 11 000 001 010 011 100 101 110 111

Other Useful Combinatorics Rules

  • There are \(n! = n \cdot (n-1) \cdot (n-2) \cdot \ldots \cdot 2 \cdot 1\) permutations of a sequence of \(n\) elements.
    • Example: \(\string{abc}\) has \(3! = 6\) permutations: \(\string{abc}\), \(\string{acb}\), \(\string{bac}\), \(\string{bca}\), \(\string{cab}\), \(\string{cba}\).
  • There are \(\binom{n}{k} = \frac{n!}{k! (n-k)!}\) ways to choose \(k\) elements from a set of size \(n\)
    (without replacement).
    • Example: number of binary strings of length 5 with exactly 2 ones:
      There are \(\binom{5}{2} = 10\) ways to choose the positions of the two ones. \[ 00011, 00101, 01001, 10001, 00110, 01010, 10010, 10100, 11000 \]
    • More examples in the lecture notes.

HW0 Submission Tutorial