CSC 173
Thurs. Dec 12, 2002
==================
  ========================================================================
Limits on Logic

Given that we can model computation as a "proof", and proceed from facts
(axioms) and rules to prove new facts (ie, compute), we are interested in
knowing whether there are any limits to what we can prove (ie, compute).

Two of the most fundamental results in mathematical logic place limits on
what we can compute:

   * The Incompleteness Theorem [Kurt Goedel, 1931] states that for
     number theory (predicate logic restricted to predicates involving
     integers, +, *, =, and <), there are expressions that are true, but
     which cannot be proved to be true.

   * The Halting Problem [Alan Turing, 1936] states that for any
     computational model equivalent to modern computers (including
     Pascal, C, and Prolog) there is no program that takes as input an
     arbitrary program and its input, and determines whether or not the
     program halts on that input.

  ------------------------------------------------------------------------
The halting problem is undecidable

Suppose we could decide, given a program and its input, whether that
program would halt when run on that input.  Then we could create the
following program, D, that takes some other program P as input, and
decides whether P halts when given itself (i.e. the text of P) as input.
If P halts, D runs it and looks at its output.  If P prints "true", D
also prints "true".  Otherwise (if P doesn't halt, or prints something
other than "true"), D prints "false".

Given D we can build a "complement" program C that also takes programs
as input.  Internally C runs D and then prints the opposite of what D
does.  That is, if D prints "true", C prints "false".  If D prints
"false", C prints "true".

Now the question arises: what does C print when given itself as input?
If C prints "false" that means D would have printed "true", meaning that
D's input program (namely C!) would have printed "true" (ouch).
Similarly, if C prints "true" that means D would have printed "false",
meaning that D's input program (C) would either have run forever or
halted and printed something other than "true" (double ouch).

The contradiction implies that our original assumption (the existence of
D) must be wrong.

  ------------------------------------------------------------------------
Number theory is undecidable

Goedel proved the incompleteness theorem before Turing proved the
halting problem.  Given the halting problem, however, we can write a
particularly nice proof of the incompleteness theorem.

Consider a program running on a computer.  The computer operates in a
series of discrete steps, each of which can be captured by the contents
of memory (including the program counter and all other registers).  Some
special value, appearing in some special subset of the memory, indicates
the act of halting.

If the program halts there must exist some maximum, over time, of the
amount of memory it uses.  Call that maximum m (expressed in bits).  We
can capture the entire step-by-step history of the program as a (huge!)
binary integer.  Start with a string of m ones, followed by a zero.
This indicates the grouping of subsequent bits.  Each subsequent group
of m bits indicates the contents of memory for the next step of the
computation.

The rules that take us from one step to another can be expressed in
terms of simple arithmetic functions.  This means we can create a
predicate P(e,p,i) that is true iff e is the encoding of a computation
of program p, running with input i.

Now suppose we that number theory is decidable.  Then every true
statement has a proof.  But now we can solve the halting problem!  Given
a program p, consider the statements (Ei)[P(e,p,i)] (the program halts)
and (Ai)[~P(e,p,i)] (the program does not halt).  One of these
statements must have a proof, which will have finite length, and which
we can check in finite time.  All (!) we have to do is enumerate all the
proofs in the world, one at a time, (e.g. starting with the shortest and
working up), until we find one for one of our two statements.  This
constitutes a (very slow) algorithm to solve the halting problem.

Since we know the halting problem is undecidable, or original assumption
(the decidability of number theory) must be wrong.