• Node and edge lists: \(\encoding{G} = \text{(binary encoding of)}\qquad (\fragment{(1,2,3,4)}, \fragment{((1,2),(2,3),(3,1),(1,4))})\)

• Adjacency matrix: \[ \fragment{\begin{bmatrix} 0 & 1 & 1 & 1 \\ 1 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ \end{bmatrix}} \fragment{\quad \Longrightarrow \quad \encoding{G} = } \fragment{0111101011001000} \hspace{15em} \]

  How can we determine the number of nodes?

  We use \(\encoding{O}\) to denote the encoding of an object \(O\) as a string in \(\binary^*\). We can encode multiple objects with \(\encoding{(O_1, O_2, \ldots, O_k)}\) or just \(\encoding{O_1, O_2, \ldots, O_k}\).

• What is the size of the adjacency matrix representation for a graph with \(n\) nodes? \(n^2\)
• What about the node+edge list representation for a graph with \(n\) nodes and \(m\) edges?
  Proportional to \(n + m\). For sparse graphs \(m \ll n^2\)…
• This sort of difference is a big deal, e.g., in high-performance computing.

Two competing algorithms:

def three_sum_1(A: list[int]) -> bool:
    for i in range(len(A)):
        for j in range(i, len(A)):
            for k in range(j, len(A)):
                if A[i] + A[j] + A[k] == 0:
                    return True
    return False

def three_sum_2(A: list[int]) -> bool:
    two_sums = set()
    for i in range(len(A)):
        for j in range(i, len(A)):
            two_sums.add(A[i] + A[j])
    for i in range(len(A)):
        if -A[i] in two_sums:
            return True
    return False

How do we know which one is faster?

Simple idea: measure the “wall-clock time” it takes to run each on the same input.

• We have experimental evidence that three_sum_2 is faster than three_sum_1 in this sense:

  Input Size	Algorithm 1 Time (s)	Algorithm 2 Time (s)
  50	0.0118399	0.0447408
  100	0.0749655	0.166695
  500	7.68075	0.377361

  These times were collected for a “worst-case input”!

• To formally define and compare the growths of time bound functions \(t(n)\), we use asymptotic analysis.

• Ignore constant factors (e.g., \(2n^2\) considered equivalent to \(3n^2\)).
• Consider only the highest-order term (e.g., \(2 n^2 + n + 2\) is equivalent to \(n^2\)).

• Polynomial bounds: \(f = O(n^c)\) for \(c > 0\) (“\(f\) is polynomially bounded”)
  • Examples: \(n^2, n^3, n^{2.2}, n^{1000}\)
  • Larger than polynomial: \(n^{\log(n)}\) (exponent is not constant; grows with \(n\))
• Exponential bounds: \(f = O(2^{c n^\delta})\) for some \(c, \delta > 0\)
  • Examples: \(2^n, 2^{100n}, 2^{0.01 n}, (2^n)^2 = 2^{2n}, 2^{n^2}, 2^\sqrt{n}, e^{n^2} (=2^{c n^2} \text{ for } c=\ln 2)\)
  • Also: \(a^{n^\delta}\) for any \(a > 1\) and \(\delta > 0\)

1. First, simplify by removing constants and lower-order terms (without changing the growth rate):
   • Examples: \[10 n^7 + 100 n^4 + n^2 + 10n = O(n^7), \quad\quad \quad\quad 2^n + n^{100} + 2^n = O(2^n) \]
   • This is helpful for the common case of an algorithm with multiple stages of different complexities.
2. Second, you might notice that \(g(n)\) is of the form \(f(n) \cdot h(n)\) with an unbounded \(h(n)\).
   • Example: \(f(n) = n, \quad g(n) = n \log(n), \fragment{\quad g(n) = f(n) \cdot \underbrace{\log(n)}_{h(n)}}\)
   • Since \(\lim\limits_{n\to\infty} \log(n) = \infty\), we conclude \(n = o(n \log(n))\).
3. Applying a log or raising to a power less than \(1\) shrinks the growth rate.
   \(\log(n) = o(n), \quad \fragment{\sqrt{n} = o(n),} \quad \fragment{\log(n^4) = o(n^4)}\) (actually \(\log(n^4) = 4 \log(n) = O(\log(n))\))
4. Try taking a log of both functions since \(\log(f(n)) = o(\log(g(n)))\) implies \(f(n) = o(g(n))\).
   • Example: compare \(n^n\) to \(2^{n^2}\) \(\log(n^n) = \fragment{n \log n},\ \log(2^{n^2}) = \fragment{n^2,} \fragment{\text{ and } n \log n = o(n^2)} \fragment{\implies n^n = o(2^{n^2})}\)
   • Warning: \(f(n) = o(g(n))\) does not imply \(\log(f(n)) = o(\log(g(n)))\) (Counterexample: \(2^n = o(2^{2n})\), but \(n \ne o(2n)\))