70

As there are non-Turing complete languages out there, and given I didn't study Comp Sci at university, could someone explain something that a Turing-incomplete language (like Coq) cannot do?

Or is the completeness/incompleteness of no real practical interest (i.e. does it not make much difference in practice)?

EDIT - I'm looking for an answer along the lines of you cannot build a hash table in a non-Turing complete language due to X, or something like that!

missingfaktor
  • 90,905
  • 62
  • 285
  • 365
oxbow_lakes
  • 133,303
  • 56
  • 317
  • 449
  • 1
    By the way, an earlier similar question had several interesting answers that are complementary to the ones here: http://stackoverflow.com/questions/315340/practical-non-turing-complete-languages – Gilles 'SO- stop being evil' Aug 16 '10 at 15:57
  • @ulidtko "Being able to interpret Turing-complete language necessitates Turing-completeness of the interpreter's host language" ... this is true. The Agda code referenced there avoids simulating the BF code. It does "assign semantics" to any BF program and could in principle be used in a framework to allow proving the termination of specific BF programs in Agda. – Atsby Apr 07 '15 at 10:24

4 Answers4

110

First, I assume you've already heard of the Church-Turing thesis, which states that anything we call “computation” is something that can be done with a Turing machine (or any of the many other equivalent models). So a Turing-complete language is one in which any computation can be expressed. Conversely, a Turing-incomplete language is one in which there is some computation that cannot be expressed.

Ok, that wasn't very informative. Let me give an example. There is one thing you cannot do in any Turing-incomplete language: you can't write a Turing machine simulator (otherwise you could encode any computation on the simulated Turing machine).

Ok, that still wasn't very informative. the real question is, what useful program cannot be written in a Turing-incomplete language? Well, nobody has come up with a definition of “useful program” that includes all the programs someone somewhere has written for a useful purpose, and that doesn't include all Turing machine computations. So designing a Turing-incomplete language in which you can write all useful programs is still a very long-term research goal.

Now there are several very different kinds of Turing-incomplete languages out there, and they differ in what they can't do. However there is a common theme. If you're designing a language, there are two major ways to ensure that the language will be Turing-complete:

  • require that the language has arbitrary loops (while) and dynamic memory allocation (malloc)

  • require that the language supports arbitrary recursive functions

Let's look at a few examples of non-Turing complete languages that some people might nonetheless call programming languages.

  • Early versions of FORTRAN did not have dynamic memory allocation. You had to figure out in advance how much memory your computation would need and allocate that. In spite of that, FORTRAN was once the most widely used programming language.

    The obvious practical limitation is that you have to predict the memory requirements of your program before running it. That can be hard, and can be impossible if the size of the input data is not bounded in advance. At the time, the person feeding the input data was often the person who had written the program, so it wasn't such a big deal. But that's just not true for most programs written today.

  • Coq is a language designed for proving theorems. Now proving theorems and running programs are very closely related, so you can write programs in Coq just like you prove a theorem. Intuitively, a proof of the theorem “A implies B” is a function that takes a proof of theorem A as an argument and returns a proof of theorem B.

    Since the goal of the system is to prove theorems, you can't let the programmer write arbitrary functions. Imagine the language allowed you to write a silly recursive function that just called itself (pick the line that uses your favorite language):

    theorem_B boom (theorem_A a) { return boom(a); }
    let rec boom (a : theorem_A) : theorem_B = boom (a)
    def boom(a): boom(a)
    (define (boom a) (boom a))
    

    You can't let the existence of such a function convince you that A implies B, or else you would be able to prove anything and not just true theorems! So Coq (and similar theorem provers) forbid arbitrary recursion. When you write a recursive function, you must prove that it always terminates, so that whenever you run it on a proof of theorem A you know that it will construct a proof of theorem B.

    The immediate practical limitation of Coq is that you cannot write arbitrary recursive functions. Since the system must be able to reject all non-terminating functions, the undecidability of the halting problem (or more generally Rice's theorem) ensures that there are terminating functions that are rejected as well. An added practical difficulty is that you have to help the system to prove that your function does terminate.

    There is a lot of ongoing research on making proof systems more programming-language-like without compromising their guarantee that if you have a function from A to B, it's as good as a mathematical proof that A implies B. Extending the system to accept more terminating functions is one of the research topics. Other extension directions include coping with such “real-world” concerns as input/output and concurrency. Another challenge is to make these systems accessible to mere mortals (or perhaps convince mere mortals that they are in fact accessible).

  • Synchronous programming languages are languages designed for programming real-time systems, that is, systems where the program must respond in less than n clock cycles. They are mainly used for mission-critical systems such as vehicle controls or signaling. These languages offer strong guarantees on how long a program will take to run and how much memory it may allocate.

    Of course, the counterpart of such strong guarantees is that you can't write programs whose memory consumption and running time you're not able to predict in advance. In particular, you can't write a program whose memory consumption or running time depends on the input data.

  • There are many specialized languages that don't even try to be programming languages and so can remain comfortably far from Turing completeness: regular expressions, data languages, most markup languages, ...

By the way, Douglas Hofstadter wrote several very interesting popular science books about computation, in particular Gödel, Escher, Bach: an Eternal Golden Braid. I don't recall whether he explicitly discusses limitations of Turing-incompleteness, but reading his books will definitely help you understand more technical material.

Gilles 'SO- stop being evil'
  • 104,111
  • 38
  • 209
  • 254
  • Nice answer. Note: I think you may mean 'popularization' instead of 'vulgarization'. (I don't think the second word has your intended meaning in English.) – Nathan Shively-Sanders Aug 17 '10 at 01:52
  • I disagree with the dynamic memory allocation requirement. Considering that all physical computers have finite memories, wouldn't all actual languages have that problem? Sure, my C program can do dynamic allocation, but that will fail on my 32-bit machine once it allocates 4GB, so how is that different from me "predicting" in advance that my program will require 4GB? – Gabe Oct 27 '10 at 04:02
  • 11
    @Gabe: You're confusing the language and its implementation. Any actual implementation on a computer will be limited by the finite memory. But that just means the implementation only handles a finite (but useful) subset of the language. Note that languages are a worthwhile study even if they're technically unimplementable because it's a lot easier to reason about an infinite system than about an extremely large finite system. – Gilles 'SO- stop being evil' Oct 27 '10 at 17:42
  • Gilles: I just don't see how dynamic memory allocation is a requirement for Turing completeness. Even if FORTRAN 77 doesn't have `malloc`, you can still write your own `malloc` using large preallocated arrays. – Gabe Oct 27 '10 at 18:08
  • 3
    @Gabe: No, you can't write `malloc` (well, an idealized `malloc` that allows you to implement a Turing-complete language) with preallocated arrays. You would still allow your program to access only a finite amount of memory; therefore from a theoretical point of view all you have is a finite automaton. – Gilles 'SO- stop being evil' Oct 27 '10 at 20:26
  • 2
    @Gabe: By the way, the Turing-completeness of C is a little subtle. Since pointers must be fully determined by their bit pattern, any given C implementation (even a conceptual one) can only provide a finite amount of memory to a program (assuming you don't use I/O to provide additional storage). However, you can make a “meta-implementation” that runs a C program in an implementation, but starts from scratch with larger implementation limits if `malloc` runs out of memory. That meta-implementation of C is a Turing-complete programming environment. – Gilles 'SO- stop being evil' Oct 27 '10 at 20:31
  • 1
    Gilles: What is it about C that makes it capable of this meta-implementation but not FORTRAN? – Gabe Oct 27 '10 at 20:40
  • @Gabe: As far as I know, early versions of FORTRAN have nothing like malloc: the size of a data structure must be specified in the program text. In C you can do something like `while (...) {n *= 2; realloc(p, n); ...}` to increase the size of the memory you can access (you will eventually run into an implementation limit, hence the need for a meta-implementation). I think (please correct me if I'm wrong) that pre-95 FORTRAN has no similar feature, and therefore is more limited (I suspect you can only write primitive recursive functions or something like that). – Gilles 'SO- stop being evil' Oct 27 '10 at 23:14
  • Gilles: Let's assume we have a FORTRAN program that creates FORTRAN programs that emulate Turing machines. Whenever the outer program detects that the inner program's array is too small it generates a new Turing machine emulator that has a larger array and restarts the program. Thus, FORTRAN is Turing complete. – Gabe Oct 28 '10 at 00:46
  • @Gabe: No, sorry, you haven't proved that FORTRAN is Turing complete; you've proved that FORTRAN (or rather a meta-FORTRAN that has no limitation on array length, thus no limitation on integer size) *plus an storage device of unbounded size* is Turing-complete. But if you're willing to use that storage device, there's a much easier Turing machine implementation: write a universal Turing machine automaton as a FORTRAN program (it's a finite automaton, so obviously expressible), and use the storage for the tape. – Gilles 'SO- stop being evil' Oct 28 '10 at 17:20
  • In the Coq item you say, "Extending the system to accept more non-terminating functions is one of the research topics." I think you mean "more (non-primitive-recursive) terminating functions"? – Chris Conway Oct 28 '10 at 17:57
  • 1
    Gilles: Isn't it the case that *all* languages require a "storage device of unbounded size" to be Turing-complete? Given that no implementation of any language is unbounded, how can any language be Turing-complete? – Gabe Oct 29 '10 at 01:21
  • 6
    @Gabe: No concrete implementation is Turing-complete, since no unbounded storage device exists. (If you only have a billion terabytes, it's not unbounded.) But some programming languages are unbounded. For example languages where the basic integer type is a bignum (such as Lisp, Haskell, Python) are straightforwardly Turing-complete. Languages that have no built-in limitation (like the one imposed in C by the fact that pointers have a given number of bits) are also easily Turing-complete (e.g. I think Java, C# fall into this category). – Gilles 'SO- stop being evil' Oct 29 '10 at 07:34
  • I think the link in "proving theorems" should not link to automated theorem proving, but interactive theorem proving, or proof assistants, as this is what Coq is doing.. – Kristopher Micinski Oct 02 '12 at 21:03
  • @KristopherMicinski Counter-quibble: The Vernacular is interactive theorem proving, but the tactic language is automated theorem proving, and the CoC-based proof language (which is what this part of my answer is about) is proof verification. I think the [Automated theorem proving](http://en.wikipedia.org/wiki/Automated_theorem_proving) is a better presentation of what such languages are about, [Proof assistant](http://en.wikipedia.org/wiki/Proof_assistant) is barely more than a comparison table. – Gilles 'SO- stop being evil' Oct 02 '12 at 21:15
  • @Gilles not true, the tactic language is only theorem proving if it does indeed terminate, which in practice it _should_, but does not by necessity. The upshot is that while the tactics are implementing nominally decidable fragments of higher order logic, saying it is automated theorem proving is disingenuous to what is really happening. – Kristopher Micinski Oct 02 '12 at 22:08
  • @Gilles however, I do agree that the proof assistant page sucks, what is needed is a page which covers interactive theorem proving, but notes that tactics should cover a decidable fragment.. – Kristopher Micinski Oct 02 '12 at 22:16
  • Great answer! Here's a comment though. *[...] you can't write programs whose memory consumption and running time you're not able to predict in advance.* — this statement is dubious to me, especially considering results of ongoing developments in type theory, including concepts like [dependent types](http://en.wikipedia.org/wiki/Dependent_type) and [codata](http://blog.sigfpe.com/2007/07/data-and-codata.html). It's hard to avoid falling into this trap, but one *has* to apply extreme caution when writing technically: with time, not rigorous enough statements may suddenly appear as wrong. – ulidtko Jan 20 '15 at 18:52
  • @ulidtko You don't need to know the exact running time (memory consumption is bounded by the running time). But in a system with a decidable static termination proof, then by construction, the termination proof gives a bound on the running time. It may be a extremely high bound, but the undecidability of the halting problem implies that in any such system, there are computations which can't be proven to terminate and thus cannot be expressed, even though they do terminate. – Gilles 'SO- stop being evil' Jan 20 '15 at 19:07
  • @Gilles right, but you didn't read the post on codata, did you? It *is* possible to write an OS in a total language. Some programmers don't even need to write terminating programs all the time; they do however need *well-behaved* programs. Generalize *termination* to *well-defined behavior* in your argument — then it suddenly fades in appeal. What about **coinduction**, a method of proof (a *tactic* in machine-assisted proving) which can be used to state equality of infinite data structures, in a rigorous and *well-defined* way? One can write a GUI event loop in a total language with codata. – ulidtko Jan 21 '15 at 10:20
  • @Gilles I think the difference is very clear to me. *"... that would be impossible"* — correct, but the conclusion I draw from this is not that "This is not an interpreter", but rather this: it's practically possible to compute full traces of execution of Turing-complete machines in a total language with codata, in a well-defined way. To call that "an interpreter" or not is a different and uninteresting question. The paramount question this gives clues for is: how far can we go in writing useful programs *without* requiring full Turing-completeness? Which computations exist *below* that class? – ulidtko Jan 21 '15 at 19:03
  • I mention all this with just one goal: to prevent further circulation of the **myth** that Turing-fit languages are not useful. They are. Yes, for *coding programs* too! (Not only for declarative data etc). We don't even really know (yet) how much useful they can be. – ulidtko Jan 21 '15 at 19:12
  • @Giles no, you do not always get bounds on running time from static termination proofs (I'm not sure what you mean by decidable): if your termination proof uses Markov's principle (roughly equivalent to assuming excluded middle in your meta-theory), then you don't get any bonds. There are also some terminating programs which are, I think, quite challenging to compute bounds on: seemingly impossible functional programs can have running time dependent on the modulus of uniform continuity of the input functions. – Jason Gross May 23 '19 at 06:37
6

The most direct answer is: a machine/language that is not Turing complete cannot be used to implement/simulate Turing machines. This comes from the basic definition of Turing completeness: a machine/language is turing complete if it can implement/simulate Turing machines.

So, what are the practical implications? Well, there is a proof that anything that can be shown to be turing complete can solve all computable problems. Which by definition means that anything that is not turing complete has the handicap that there are some computable problems that it can't solve. What those problems are depends on what features are missing that makes the system non-Turing complete.

For example if a language doesn't support looping or recursion or implicitly loops cannot be Turing complete because Turing machines can be programmed to run forever. Consequently that language cannot solve problems requiring loops.

Another example is if a language doesn't support lists or arrays (or allow you to emulate them for example using the filesystem) then it cannot implement a Turing machine because Turing machines require arbitrary random access to memory. Consequently that language cannot solve problems requiring arbitrary random access to memory.

So, the feature that is missing that classifies a language to be non-Turing complete is the very thing that practically limits the usefulness of the language. So the answer is, it depends: what makes the language non-Turing complete?

slebetman
  • 109,858
  • 19
  • 140
  • 171
  • +1 You can't write an interpreter for a turing-complete language in a non-turing complete language. You also can't do anything equivalent to that (where equivalent means that writing an interpreter for a turing-complete language is reducible to that problem). For every thing else (that is computable) there exists a non-turing complete language where you can do it. – sepp2k Aug 16 '10 at 11:04
  • How would I "prove" that some given problem *required* loops to be solved. For example, it may be the case that anything solvable using loops is also solvable using recursion? – oxbow_lakes Aug 16 '10 at 11:48
  • @oxbow_lakes: It is provable that recursion and looping are interchangeable. Therefore, by loop we mean recursion as well. No looping means unable to do recursion. – slebetman Aug 16 '10 at 12:17
  • 5
    @oxbow_lakes: Approach 1: to prove that problem P requires loops to be solved in language L, you define a language K that is L without loops and show that problem P cannot be solved in language K. Typically that last part is done by proving that every program in language K has a certain property (e.g., a bound on running time), and that problem P does not have that property. – Gilles 'SO- stop being evil' Aug 16 '10 at 12:19
  • 4
    @oxbow_lakes: Approach 2: to prove that problem P requires *loops or some equivalent feature* to be solved, you try to encode loops into problem P. For example, to prove that having loops requires recursion (i.e., that loops are as powerful as recursion), you show that any recursive program can be written using loops. – Gilles 'SO- stop being evil' Aug 16 '10 at 12:22
6

An important class of problems that are a bad fit for languages such as Coq is those whose termination is conjectured or hard to prove. You can find plenty of examples in number theory, maybe the most famous is the Collatz conjecture

function collatz(n)
  while n > 1
    if n is odd then
      set n = 3n + 1
    else
      set n = n / 2
    endif
 endwhile

This limitation leads to have to express such problems in a less natural way in Coq.

ejgallego
  • 6,709
  • 1
  • 14
  • 29
  • 5
    I disagree: you can perfectly well express Collatz conjecture in Coq: `forall n, exists k, (iterate collatz k) n = 1.` (where `iterate f k` applies a function `k` times). – Olivier Verdier Dec 07 '16 at 17:18
  • @OlivierVerdier indeed, but note that you are writing down the conjecture vs the function directly, for many people this is less natural. I never claimed you couldn't express the conjecture perfectly well. – ejgallego Dec 07 '22 at 19:13
3

You can't write a function that simulates a Turing machine. You can write a function that simulates a Turing machine for 2^128 (or 2^2^2^2^128 steps) and reports whether the Turing machine accepted, rejected, or ran for longer than the allowed number of steps.

Since "in practice" you will be long gone before your computer can simulate a Turing machine for 2^128 steps, it's fair to say that Turing incompleteness does not make much of a difference "in practice".

Atsby
  • 2,277
  • 12
  • 14