Title

ECS 120 Theory of Computation
Introduction
Julian Panetta
University of California, Davis

ECS 120 Teaching Team

data/images/people/julian_300x300.pngJulian data/images/people/johnson.jpgJohnson data/images/people/haleh.jpgHaleh

Acknowledgements

  • Massive thanks to David Doty!
    • This course is based on the lecture materials, assessments, and policies he’s developed over the years starting back in 2015.
    • We’ll be using several tools he’s developed like the automaton simulator for simulating various computational models and his autograding infrastruture for the homeworks.
    • He’s been incredibly generous in sharing these materials!
data/images/people/dave_gray.jpg
David Doty

General Information

Lectures

  • I will upload slides before the start of class so you can annotate them during lecture.
  • Please interrupt at any time to ask questions!
  • Attendance is not required except on midterm day… but it’s strongly recommended!
  • Recording of lectures is not planned this quarter.
    • … but we’re closely following Dave Doty’s lectures, which have been recorded.
    • The relevant recording(s) for each lecture will be linked from the Canvas page.

How to use the HTML slides

  • Use the keys left/right to navigate through the slides.
  • Click page number (bottom right) to open navigation menu.
  • Press f to enter fullscreen mode, ESC to leave fullscreen mode.
  • Double-click an item (e.g. an image) to zoom in/out.
  • The slides are best viewed with Chrome or Chromium. Firefox unfortunately has some problems with vector graphics and doesn’t support some video codecs.

Live Polls/“Clicker Questions”

What is your major?

  • Computer Science
  • Electrical and Computer Engineering
  • Mathematics
  • Other

Live Polls/“Clicker Questions”

Which of the recommended prerequisites have you taken?

  • ECS 32B
  • ECS 36C
  • Neither

Theory of Computation

  • Fundamental questions:
    • What is a computer? How can we model computers formally to prove things about them?
      data/images/intro/eniac.jpgENIAC – 1945 data/images/intro/computer.pngPC – c. 2000 data/images/intro/IPhone_1st_Gen.svgiPhone – 2007 data/images/intro/example_nfa.svgFinite Automata – 1943 data/images/intro/turing_machine.svgTuring Machines – 1936
    • What is a computational problem?
    • What is an algorithm?
    • Which problems can and can’t be solved by a computer/algorithm?
    • Which problems can be solved efficiently?
  • This course will build up the tools to answer these questions rigorously.

… but aren’t the answers to these already intuitively clear?

Motivating Example: Polynomial Equations

  • Consider polynomial equations with integer coefficients:

    1. \(x^2 - 2x + 1 = 0\)
    2. \(x^2 - 2x - 1 = 0\)
    3. \(x^2 + 1 = 0\)
    4. \(x^2 - y^2 - 2 = 0\)
    5. \(x^2 + 2 x y - y^3 - 13 = 0\)
      …

  • Which of these has at least one solution (root) of a particular type?
    Specifically, we are interested in the following cases:

    Solution Type

    \(\R\) (Real)

    \(\Z\) (Integer)

    # of Vars

    1
    ?
    ?

    \(\ge 2\)
    ?
    ?

Motivating Example: Polynomial Equations

  • Consider polynomial equations with integer coefficients:

    1. \(x^2 - 2x + 1 = 0\)
    2. \(x^2 - 2x - 1 = 0\)
    3. \(x^2 + 1 = 0\)
    4. \(\color{lightgrey} x^2 - y^2 - 2 = 0\)
    5. \(\color{lightgrey} x^2 + 2 x y - y^3 - 13 = 0\)
      …

  • Which of these has at least one solution (root) of a particular type?
    Specifically, we are interested in the following cases:

    Solution Type

    \(\R\) (Real)

    \(\Z\) (Integer)

    # of Vars

    1
    ?
    ?

    \(\ge 2\)
    ?
    ?
    • These three examples are all easy (quadratic formula):
      1. \(x = \frac{2 \pm \sqrt{4 - 4}}{2} = 1 \pm 0\) (1 integer root)
      2. \(x = \frac{1 \pm \sqrt{4 + 4}}{2} = 1 \pm \sqrt{2}\) (2 real roots)
      3. \(x = \frac{\pm \sqrt{-4}}{2} = \pm i\) (2 complex roots)

Motivating Example: Polynomial Equations

  • Consider polynomial equations with integer coefficients:

    1. \(\color{lightgrey} x^2 - 2x + 1 = 0\)
    2. \(\color{lightgrey} x^2 - 2x - 1 = 0\)
    3. \(\color{lightgrey} x^2 + 1 = 0\)
    4. \(x^2 - y^2 - 2 = 0\)
    5. \(x^2 + 2 x y - y^3 - 13 = 0\)
      …

  • Which of these has at least one solution (root) of a particular type?
    Specifically, we are interested in the following cases:

    Solution Type

    \(\R\) (Real)

    \(\Z\) (Integer)

    # of Vars

    1
    (a), (b)
    (b)

    \(\ge 2\)
    ?
    ?
    • Real solutions to \(x^2 - y^2 - 2 = 0\) are easy to find: \[ \setbuild{(\pm \sqrt{2 + y^2}, y)}{y \in \R}% = \setbuild{(x, \pm \sqrt{x^2 - 2})}{x \ge \sqrt{2}} \] data/images/intro/roots/x2-y2-2_roots.svg
    • What about solutions where \(x\) and \(y\) are both integers? data/images/intro/roots/x2-y2-2_roots_integer.svg
      … I don’t see any…
    • We can prove none exist by realizing \(x^2 = y^2 + 2\) is asking for squares of integers that are two apart, but the perfect squares are \(\{0, 1, 4, 9, 16, \ldots\}\).

Motivating Example: Polynomial Equations

  • Consider polynomial equations with integer coefficients:

    1. \(\color{lightgrey} x^2 - 2x + 1 = 0\)
    2. \(\color{lightgrey} x^2 - 2x - 1 = 0\)
    3. \(\color{lightgrey} x^2 + 1 = 0\)
    4. \(x^2 - y^2 - 2 = 0\)
    5. \(x^2 + 2 x y - y^3 - 13 = 0\)
      …

  • Which of these has at least one solution (root) of a particular type?
    Specifically, we are interested in the following cases:

    Solution Type

    \(\R\) (Real)

    \(\Z\) (Integer)

    # of Vars

    1
    (a), (b)
    (b)

    \(\ge 2\)
    (d), ?
    ?
    • What about \(x^2 + 2 x y - y^3 - 13 = 0\)? data/images/intro/roots/x2-2xy-y3-13_roots.svg
    • Real solutions clearly can be found for any \(x \in \R\).
    • What about integer solutions? data/images/intro/roots/x2-2xy-y3-13_roots_integer.svg Actually, yes!
      For example, \((x, y) = (3, 2)\) is a solution.

Motivating Example: Polynomial Equations

  • Consider polynomial equations with integer coefficients:

    1. \(\color{lightgrey} x^2 - 2x + 1 = 0\)
    2. \(\color{lightgrey} x^2 - 2x - 1 = 0\)
    3. \(\color{lightgrey} x^2 + 1 = 0\)
    4. \(x^2 - y^2 - 2 = 0\)
    5. \(x^2 + 2 x y - y^3 - 13 = 0\)
      …

  • Which of these has at least one solution (root) of a particular type?
    Specifically, we are interested in the following cases:

    Solution Type

    \(\R\) (Real)

    \(\Z\) (Integer)

    # of Vars

    1
    (a), (b)
    (b)

    \(\ge 2\)
    (d), (e)
    (e)
    • What about \(x^2 + 2 x y - y^3 - 13 = 0\)?
    • Real solutions clearly can be found for any \(x \in \R\).
    • What about integer solutions? data/images/intro/roots/x2-2xy-y3-13_roots_integer.svg Actually, yes!
      For example, \((x, y) = (3, 2)\) is a solution.

    We need a more systematic approach!

Motivating Example: Polynomial Equations

Let’s write a program that, given one of these polynomials, reports if roots exist!
Solution Type
\(\R\) (Real) \(\Z\) (Integer)
# of Vars 1
Program?
Program?
\(\ge 2\)
Program?
Program?

Motivating Example: Polynomial Equations

Let’s write a program that, given one of these polynomials, reports if roots exist!
Solution Type
\(\R\) (Real) \(\Z\) (Integer)
# of Vars 1
Program?
Program?
\(\ge 2\)
Program?
Program?
  • Integer solutions seem easiest to find, so let’s start with those in the single-variable case.

    • Algorithm: Starting at \(x = 0\), check integers of progressively increasing magnitude until we find a root!

      ↑
      return true when we find one!

      • Wait, how do we know if/when to give up the search?

      • Here’s a trick: any root \(|x| \ge 1\) must satisfy \[ \small \begin{aligned} c_n &x^n + c_{n-1} x^{n-1} + \ldots + c_1 x + c_0 = 0 \\ \iff \quad &x^n = -\frac{1}{c_n}\left( c_{n-1} x^{n-1} + \ldots + c_1 x + c_0\right) \\ \iff \quad &x = -\frac{1}{c_n}\left( c_{n-1} + \ldots + \frac{c_1 }{x^{n - 2}} + \frac{c_0}{x^{n - 1}}\right) \\ \implies \quad &|x| = \frac{1}{|c_n|}\left| c_{n-1} + \ldots + \frac{c_1 }{x^{n - 2}} + \frac{c_0}{x^{n - 1}}\right| \le (n - 1) \frac{|c_\text{max}|}{|c_n|} \end{aligned} \hspace{24em} \]

        ← return false if we exceed this magnitude!

Motivating Example: Polynomial Equations

Let’s write a program that, given one of these polynomials, reports if roots exist!
Solution Type
\(\R\) (Real) \(\Z\) (Integer)
# of Vars 1
Program?
✅
\(\ge 2\)
Program?
Program?
  • Off to a great start! Now what about real roots of univariate polynomials?
  • This is not obvious, but an algorithm was proven to exist [Tarski, 1948].

Motivating Example: Polynomial Equations

Let’s write a program that, given one of these polynomials, reports if roots exist!
Solution Type
\(\R\) (Real) \(\Z\) (Integer)
# of Vars 1
✅
✅
\(\ge 2\)
✅
Program?
  • Off to a great start! Now what about real roots of univariate polynomials?
  • This is not obvious, but an algorithm was proven to exist [Tarski, 1948].
  • In fact, the proof works for the multi-variable case too!
  • Finally, what about the multivariate integer case? This was Hilbert’s 10th problem:
    find an algorithm for determining whether a diophantine equation has solutions.
    • Hilbert apparently thought in 1900 that it could be done…
    • 70 years later, we discovered this is impossible [Matiyasevich–Robinson–Davis–Putnam theorem]!
    • There’s no way to bound solution magnitudes like in 1D…

Motivating Example: Polynomial Equations

Let’s write a program that, given one of these polynomials, reports if roots exist!
Solution Type
\(\R\) (Real) \(\Z\) (Integer)
# of Vars 1
✅
✅
\(\ge 2\)
✅
❌

It’s not always obvious whether a problem can even be solved by an algorithm!

Motivating Example: Cycle-Finding Algorithms

Now consider the problem of identifying if a connected graph contains an Eulerian cycle.
(A path starting and ending at the same node that traverses every edge exactly once)

✅ Has an Eulerian cycle!

?

❌ Turns out not to have an
     Eulerian cycle.

Simple test: an Eulerian cycle exists if and only if all nodes have even degree!

Motivating Example: Cycle-Finding Algorithms

Similarly, we might want to check whether a graph contains a Hamiltonian cycle.
(A path starting and ending at the same node that traverses every other node exactly once)

✅ Has an Hamiltonian cycle!

❌ Turns out not to have an
     Hamiltonian cycle.

No simple test is known for whether a graph is “Hamiltonian”!

Motivating Example: Cycle-Finding Algorithms

Task:

Write an efficient algorithm to check if a graph \(G\) has a cycle visiting each edge exactly once.

Algorithm:

Check if \(G\) is connected (BFS) and every node has even degree.

Task:

Write an efficient algorithm to check if a graph \(G\) has a cycle visiting each node exactly once.

Fact:

This task is impossible (assuming P ≠ NP)!

Motivating Example: Regular Expressions

A regular expression describes a pattern that matches some strings but not others.

For example, let’s write patterns for ternary strings over the alphabet \(\Sigma = \{0, 1, 2\}\).

  • Match 0 followed by any number of 0s, 1s, and 2s: \[0(0 \cup 1 \cup 2)^*\] (matches, e.g., 010201, 0210021, 0000, 0111, 0222, 0)
  • Match any number of 0s or 2s followed by a single 1. \[(0 \cup 2)^* 1\] (matches, e.g., 202201, 000001, 22221, 1)
  • Match either of the above patterns: \[(0(0 \cup 1 \cup 2)^*) \cup ((0 \cup 2)^* 1)\]
  • String must either start with 0 or end in 1.
  • If it doesn’t start with a 0, then the last character is the only 1 in the string.

Motivating Example: Regular Expressions

Let \(\#_0(w)\) and \(\#_1(w)\) denote the number of 0s and 1s in a string \(w\), respectively.

Can we write a regular expression that matches a string \(w\) exactly when \(\#_0(w) + \#_1(w) = 2\)?

  • Yes!
  • No!

The expression is: \[ 2^* (0 \cup 1) 2^* (0 \cup 1) 2^* \]

Motivating Example: Regular Expressions

Let \(\#_0(w)\) and \(\#_1(w)\) denote the number of 0s and 1s in a string \(w\), respectively.

Can we write a regular expression that matches a string \(w\) exactly when \(\#_0(w) - \#_1(w) = 2\)?

  • Yes!
  • No!

This is impossible!

Where this Course Sits

ECS 36 A/B/C

  • Covered how to program computers
  • … but not how to reason about them as formal mathematical objects.

ECS 20

  • covered formal mathematical objects (sets, functions, relations)
  • … but those objects weren’t computers.

ECS 122

  • Find the most efficient possible algorithm to solve a problems.
    “Focus on the possible.”

ECS 120

  • Use tools of ECS 20 to study computers as formal mathematical objects.
  • Prove that no (efficient) algorithm exists for certain problems.

Main Topics

  1. Automata Theory
    What problems can be solved with optimal efficiency?
    (with constant memory and doing constant work per bit)

  2. Complexity Theory
    What problems can computers solve efficiently?
    (In polynomial time: for \(n\)-bit inputs # of steps = \(O(n)\) or \(O(n^2)\), \(O(n^3)\), etc.)

  3. Computability Theory
    What problems can computers solve?

← First half of the course.

Course Policies and Logistics

  • Our class will use three different platforms:
    • Canvas
      • syllabus, schedule, links to lecture materials
      • homework handouts but not submission.
      • reading quizes and take-home quizzes.
    • Gradescope for homework submission.
    • Piazza for questions and discussion.
      • All discussion should take place on Piazza!
      • Send personal messages either via Piazza or email.
      • Extra credit will be awarded to students who answer questions on Piazza.

You will need to register for Piazza!
(Instructions on Canvas)

  • Discussion sessions are interactive: please show up with questions!

Grading

Assessment Type Weight
Reading Quizzes 10%
Auto-graded Homework 25%
Written Homework 5%
Take-home Quizzes 20%
Midterm Exam 20%
Final Exam 20%
Piazza Extra Credit 3%

Reading Quizzes (10%)

  • Roughly one per lecture.
  • Open book, open notes, and mostly just easy comprehension questions.
  • You’ll get two submission attempts, and your five lowest scores are dropped.
  • The goal is to make you think about the course material every week
    and avoid last-minute cramming for the homework/exam
  • Released at the beginning and due at the end of each week.

Homework (30%)

  • There will be four primary homework assignments (HW1-HW4) each with two parts:

    1. Auto-graded
      You will submit solutions (e.g., numerical answers or a finite automaton specification) to Gradescope and receive immediate feedback on correctness, with opportunities for resubmission.

    2. Written
      You will submit proofs or written arguments also to Gradescope, but these will be graded by the TAs for completeness only. You must make a serious attempt for credit (and give an answer of the “correct type”).

  • The written part is mainly preparation for the take-home quizzes and in-person exams.

Take-home Quizzes (20%)

  • After HW1-HW4 are due, you will complete a 30 minute timed, open-notes (but no internet or collaboration) quiz on Canvas.
  • The content will mostly resemble that of the associated homework and reading quizzes.
  • The lowest quiz score will be dropped.

Exams (40%)

  • There will be two in-person exams
    • Midterm during one of the lecture
    • Final at 8am on June 12
  • The midterm will cover Automata Theory (up through Chapter 7 of the lecture notes)
  • The final exam will cover only complexity and computability theory (not cumulative)
  • Exams will be closed-book but allow a 3x5” handwritten notecard.

Late Penalties for Homework

  • The take-home quiz deadlines are firm (the quiz cannot be attempted after the deadline).

  • Homework can be submitted up to 24 hours late with a continuously increasing penalty.
    If you earn a score of \(s\) on the assignment, your final score will be: \[ s \times \max\left(1 - \left(\frac{h}{24}\right)^4, 0\right) \] data/images/late-penalty.png

    Example penalized scores for
    a perfect submission (\(s = 100\%\)):

    Submission Time Score (%)
    On time 100.00%
    1 hour late 100.00%
    6 hours late 99.61%
    12 hours late 93.75%
    18 hours late 68.36%
    23 hours late 15.65%
    24 hours late 0.00%

Collaboration Policy

  • You are encouraged to discuss course material with classmates (especially on Piazza!).
  • You may discuss solutions to the written parts and the randomized autograded problems.
  • You must not discuss or share solutions to the non-randomized autograded problems.
  • All submissions (whether written or autograded) must still be your own work.
    • Do not reference any notes or solutions from your discussions when writing up your submission.
    • If you discuss problems with another student or consult with outside sources for the written parts, cite this on your submission.
  • No collaboration whatsoever on the take-home quizzes.

HW 0

  • HW0 is due Friday and covers some discrete math concepts.
  • ECS 20 (Discrete Math) is a pre-requisite for this course. But maybe…
    • you took it a long time ago.
    • slightly different terminology/notation was used.
  • This week’s discussion session will cover:
    • review of discrete math (Chapter 1 of the lecture notes)
    • how to use Gradescope to submit HW0