Home >Education >Metabiology life as evolving software by g j chaitin

Metabiology life as evolving software by g j chaitin

Date post:01-Sep-2014
View:971 times
Download:4 times
Share this document with a friend
Metabiology Life as Evolving Software by G J Chaitin
  • 1 METABIOLOGY: LIFE AS EVOLVING SOFTWARE METABIOLOGY: a eld parallel to biology, dealing with the random evolution of arti- cial software (computer programs) rather than natural software (DNA), and simple enough that it is possible to prove rigorous theorems or formulate heuristic arguments at the same high level of precision that is common in the- oretical physics.
  • 2 The chance that higher life forms might have emerged in this way [by Darwinian evolution] is comparable to the chance that a tornado sweeping through a junkyard might assemble a Boeing 747 from the materials therein. Fred Hoyle. In my opinion, if Darwins theory is as simple, fundamental and basic as its adherents believe, then there ought to be an equally fundamental mathemati- cal theory about this, that expresses these ideas with the generality, precision and degree of abstractness that we are accustomed to demand in pure math- ematics. Gregory Chaitin, Speculations on Biology, Information and Complexity. Mathematics is able to deal successfully only with the simplest of situations, more precisely, with a complex situation only to the extent that rare good fortune makes this complex situation hinge upon a few dominant simple fac- tors. Beyond the well-traversed path, mathematics loses its bearings in a jungle of unnamed special functions and impenetrable combinatorial partic- ularities. Thus, the mathematical technique can only reach far if it starts from a point close to the simple essentials of a problem which has simple essentials. That form of wisdom which is the opposite of single-mindedness, the ability to keep many threads in hand, to draw for an argument from many disparate sources, is quite foreign to mathematics. Jacob Schwartz, The Pernicious Inuence of Mathematics on Science. It may seem natural to think that, to understand a complex system, one must construct a model incorporating everything that one knows about the system. However sensible this procedure may seem, in biology it has repeat- edly turned out to be a sterile exercise. There are two snags with it. The rst is that one nishes up with a model so complicated that one cannot understand it: the point of a model is to simplify, not to confuse. The sec- ond is that if one constructs a suciently complex model one can make it do anything one likes by ddling with the parameters: a model that can predict anything predicts nothing. John Maynard Smith & Eors Szathmary, The Origins of Life.
  • 3 Course Notes METABIOLOGY: LIFE AS EVOLVING SOFTWARE G. J. Chaitin Draft October 1, 2010
  • 4 To my wife Virginia who played an essential role in this research
  • Contents Preface 7 1 Introduction: Building a theory 9 2 The search for the perfect language 19 3 Is the world built out of information? Is everything soft- ware? 39 4 The information economy 45 5 How real are real numbers? 55 6 Speculations on biology, information and complexity 77 7 Metaphysics, metamathematics and metabiology 87 8 Algorithmic information as a fundamental concept in physics, mathematics and biology 101 9 To a mathematical theory of evolution and biological creativ- ity 113 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 9.2 History of Metabiology . . . . . . . . . . . . . . . . . . . . . . 114 9.3 Modeling Evolution . . . . . . . . . . . . . . . . . . . . . . . . 116 9.3.1 Software Organisms . . . . . . . . . . . . . . . . . . . . 116 9.3.2 The Hill-Climbing Algorithm . . . . . . . . . . . . . . 116 9.3.3 Fitness . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 9.3.4 What is a Mutation? . . . . . . . . . . . . . . . . . . . 117 5
  • 6 9.3.5 Mutation Distance . . . . . . . . . . . . . . . . . . . . 117 9.3.6 Hidden Use of Oracles . . . . . . . . . . . . . . . . . . 118 9.4 Model A (Naming Integers) Exhaustive Search . . . . . . . . . 119 9.4.1 The Busy Beaver Function . . . . . . . . . . . . . . . . 119 9.4.2 Proof of Theorem 1 (Exhaustive Search) . . . . . . . . 119 9.5 Model A (Naming Integers) Intelligent Design . . . . . . . . . . 120 9.5.1 Another Busy Beaver Function . . . . . . . . . . . . . 120 9.5.2 Improving Lower Bounds on . . . . . . . . . . . . . . 121 9.5.3 Proof of Theorem 2 (Intelligent Design) . . . . . . . . . 123 9.6 Model A (Naming Integers) Cumulative Evolution at Random . 124 9.7 Model B (Naming Functions) . . . . . . . . . . . . . . . . . . . 128 9.8 Remarks on Model C (Naming Ordinals) . . . . . . . . . . . . . 132 9.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 10 Parsing the Turing test 143 11 Should mathematics be done dierently because of Godels incompleteness theorem? 149 Bibliography 159 Books by Chaitin 161
  • Preface Biology and mathematics are like oil and water, they do not mix. Never- theless this course will describe my attempt to express some basic biological principles mathematically. Ill try to explain the raison detre of what I call my metabiological approach, which studies randomly evolving computer programs rather than biological organisms. I want to thank a number of people and organizations for inviting me to lecture on metabiology; the interaction with audiences was extremely stimu- lating and helped these ideas to evolve. Firstly, I thank the IBM Watson Research Center, Yorktown Heights, where I gave two talks on this, including the world premiere talk on metabiol- ogy. Another talk on metabiology in the United States was at the University of Maine. In Argentina I thank Veronica Becher of the University of Buenos Aires and Victor Rodriguez of the University of Cordoba for their kind invitations. And I am most grateful to the University of Cordoba, currently celebrating its 400th anniversary, for the honorary doctorate that they were kind enough to bestow on me. In Chile I spoke on metabiology several times at the Valparaiso Complex Systems Institute, and in Brazil I included metabiology in courses I gave at the Federal University of Rio de Janeiro and in a talk at the Federal University in Niteroi. Furthermore I thank Bernd-Olaf Kuppers for inviting me to a very stim- ulating meeting at his Frege Centre for Structural Sciences at the University of Jena. And I thank Ilias Kotsireas for organizing a Chaitin-in-Ontario lecture se- ries in 2009 in the course of which I spoke on metabiology at the University of Western Ontario in London, at the Institute for Quantum Computing in Waterloo, and at the Fields Institute at the University of Toronto. The chap- 7
  • 8 Chaitin: Metabiology ter of this book on is based on a talk I gave at Wilfrid Laurier University in Waterloo. Finally, I should mention that the chapter on The Search for the Perfect Language was rst given as a talk at the Hebrew University in Jerusalem in 2008, then at the University of Campinas in Brazil, and nally at the Perimeter Institute in Waterloo, Canada. The chapter on Is Everything Software? was originally a talk at the Technion in Haifa, where I also spoke on metabiology at the University of Haifa, one of a series of talks I gave there as the Rothschild Distinguished Lecturer for 2010. These were great audiences, and their questions and suggestions were extremely valuable. Gregory Chaitin, August 2010
  • Chapter 1 Introduction: Building a theory This is a course on biology that will spend a lot of time discussing Kurt Godels famous 1931 incompleteness theorem on the limits of formal mathematical reasoning. Why? Because in my opinion the ultimate historical perspective on the signicance of incompleteness may be that Godel opens the door from mathematics to biology. We will also spend a lot of time discussing computer programs and software for doing mathematical calculations. How come? Because DNA is presumably a universal programming language, which is a language that is rich enough that it can express any algorithm. The fact that DNA is such a powerful programming language is a more fundamental characteristic of life than mere self-reproduction, which anyway is never exactfor if it were, there would be no evolution. Now a few words on the kind of mathematics that we shall use in this course. Starting with Newton mathematical physics is full of what are called ordinary dierential equations, and starting with Maxwell partial dierential equations become more and more important. Mathematical physics is full of dierential equations, that is, continuous mathematics. But that is not the kind of mathematics that we shall use here. The secret of life is not a dierential equation. There is no dierential equation for your spouse, for an organism, or for biological evolution. Instead we shall concentrate on the fact that DNA is the software, its the programming language for life. It is true that there are (ordinary) dierential equations in a highly suc- 9
  • 10 Chaitin: Metabiology cessful mathematical theory of evolution, Wright-Fisher-Haldane pop- ulation genetics. But population genetics does not say where new genes come from, it assumes a xed gene pool and discusses the change of gene frequencies in response to selective pressure, not bio- logical creativity and the major transitions in evolution, such as the transition from unicellular to multicellular organisms, which is what interests us. If we arent going to use anymore the dierential equations that popu- late mathematical physics, what kind of math are we going to use? It will be discrete math, new math, the math of the 20th century dealing with computation, with algorithms. It wont be traditional continuous math, it wont be the calculus. As Dorothy says in The Wizard of Oz, Toto, were not in Kansas anymore! More in line with our life-as-evolving-software viewpoint are three hot new topics in 20th century mathematics, computation, information and complexity. These have expanded into entire theories, called com- putability theory, information theory and complexity theory, theories which supercially appear to have little or no connection with biology. In particular, our basic tool in this course will be algorithmic infor- mation theory (AIT), a mixture of Turing computability theory with Shannon information theory, which features the concept of program- size complexity. The author was one of the people who created this theory, AIT, in the mid 1960s and then further developed it in the mid 1970s; the theory of evolution presented in this course could have been done thenall the necessary tools were available. Why then the delay of 35 years? My apologies; I got distracted working on computer engineering and thinking about metamathematics. I had published notes on biology occasionally on and o since 1969, but I couldnt nd the right way of thinking about biology, I couldnt gure out how to formulate evolution mathematically in a workable manner. Once I discovered the right way, this new theory I call metabiology went from being a gleam in my eye to a full-edged mathematical theory in just two years. Also, it would be nice to be able to show that in our toy model hierar- chical structure will evolve, since that is such a conspicuous feature of biological organisms.
  • Introduction: Building a theory 11 What kind of math can we use for that? Well, there are places in pure math and in software engineering where you get hierarchical structures: in Mandelbrot fractals, in Cantor transnite ordinal numbers, in hier- archies of fast growing functions, and in software levels of abstraction. Fractals are continuous math and therefore not suitable for our discrete models, but the three others are genuine possibilities, and we shall discuss them all. One of our models of evolution does provably exhibit hierarchical structure. Here is the big challenge: Biology is extremely complicated, and every rule has exceptions. How can mathematics possibly deal with this? We will outline an indirect way to deal with it, by studying a toy model I call metabiology (= life as evolving software, computer program organisms, computer program mutations), not the real thing. We are using Leibnizian math, not Newtonian math. By modeling life as software, as computer programs, we get a very rich space of possible designs for organisms, and we can discuss biological creativity = where new genes come from (where new biological ideas such as multicellular organization come from), not just changes in gene frequencies in a population as in conventional evolutionary models. Some simulations of evolution on the computer (in silicoas contrasted with in vivo, in the organism, and in vitro, in the test tube) such as Tierra and Avida do in fact model organisms as software. But in these models there is only a limited amount of evolution followed by stagnation.1 Furthermore I do not run my models on a computer, I prove theorems about them. And one of these theorems is that evolution will con- tinue indenitely, that biological creativity (or what passes for it in my model) is endless, unceasing. The main theme of Darwinian evolution is competition, survival of the ttest, Nature red in tooth and claw. The main theme of my model is creativity: Instead of a population of individuals competing 1 As for genetic algorithms, they are intended to stagnate when they achieve an optimal solution to an engineering design problem; such a solution is a xed point of the process of simulated evolution used by genetic algorithms.
  • 12 Chaitin: Metabiology ferociously with each other in order to spread their individual genes (as in Richard Dawkins The Selsh Gene), instead of a jungle, my model is like an individual Buddhist trying to attain enlightenment, a monk who is on the path to enlightenment, it is like a mystic or a kabbalist who is trying to get closer and closer to God. More precisely, the single mutating organism in my model attains greater and greater mathematical knowledge by discovering more and more of the bits of , which is, as we shall see in the part of the course on , Course Topic 5, a very concentrated form of mathematical knowledge, of mathematical creativity. My organisms strive for greater mathematical understanding, for purely mathematical enlightenment. I model where new mathematical knowledge is coming from, where new biological ideas are coming from, it is this process that I model and prove theorems about. But my model of a single mutating organism is indeed Darwinian: I have a single organism that is subjected to completely random muta- tions until a tter organism is found, which then replaces my original organism, and this process continues indenitely. The key point is that in my model progress comes from combining random mutations and having a tness criterion (which is my abstract encapsulation of both competition and the environment). The key point in Darwins theory was to replace God by randomness; organisms are not designed, they emerge at random, and that is also the case in my highly simplied toy model. Does this highly abstract game have any relevance to real biology? Probably not, and if so, only very, very indirectly. It is mathematics itself that benets most, because we begin to have a mathematical theory inspired by Darwin, to have mathematical concepts that are inspired by biology. The fact that I can prove that evolution occurs in my model does not in any way constitute a proof that Darwinians are correct and Intelligent Design partidarians are mistaken. But my work is suggestive and it does clarify some of the issues, by furnishing a toy model that is much easier to analyze than the real thingthe real thing is what is actually taking place in the biosphere,
  • Introduction: Building a theory 13 not in my toy model, which consists of arbitrary mutation computer programs operating on arbitrary organism computer programs. More on creativity, a key word in my model: Something is mechanical if there is an algorithm for doing it; it is creative if there is no such algorithm. This notion of creativity is basic to our endeavor, and it comes from the work of Godel on the incompleteness of formal axiomatic theories, and from the work of Turing on the unsolvability of the so-called halt- ing problem.2 Their work shows that there are no absolutely general methods in mathematics and theoretical computer science, that cre- ativity is essential, a conclusion that Paul Feyerabend with his book Against Method would have loved had he been aware of it: What Fey- erabend espouses for philosophical reasons is in fact a theorem, that is, is provably correct in the eld of mathematics. So before we get to Darwin, we shall spend a lot of time in this course with Godel and Turing and the like, preparing the groundwork for our model of evolution. Without this historical background it is impossible to appreciate what is going on in our model. My model therefore mixes mathematical creativity and biological cre- ativity. This is both good and bad. Its bad, because it distances my model from biology. But it is good, because mathematical creativity is a deep mathematical question, a fundamental mystery, a big un- known, and therefore something important to think about, at least for mathematicians, if not for biologists. Further distancing my model from biology, my model combines ran- domness, a very Darwinian feature, with Turing oracles, which have no counterpart in biology; we will discuss this in due course. Exploring such models of randomly evolving software may well develop into a new eld of mathematics. Hopefully this is just the beginning, and metabiology will develop and will have more connection with biol- ogy in the future than it has at present. 2 Turings halting problem is the question of deciding whether or not a computer pro- gram that is self-contained, without any input, will run forever, or will eventually nish.
  • 14 Chaitin: Metabiology The main dierence between our model and the DNA software in real organisms is their time complexity: the amount of time the software can run. I can prove elegant theorems because in my model the time allowed for a program to run is nite, but unlimited. Real DNA software must run quickly: 9 months to produce a baby, 70 years in total, more or less. A theory of the evolution of programs with such limited time complexity, with such limited run time, would be more realistic but it will not contain the neat results we have in our idealized version of biology. This is similar to the thermodynamics arguments which are taken in the thermodynamic limit of large amounts of time, in order to obtain more clear-cut results, when discussing the ideal performance of heat engines (e.g., steam engines). Indeed, AIT is a kind of thermodynamics of computation, with program-size complexity replacing entropy. In- stead of applying to heat engines and telling us their ideal eciency, AIT does the same for computers, for computations. You have to go far from everyday biology to nd beautiful mathematical structure. It should be emphasized that metabiology is Work in Progress. It may be mistaken. And it is certainly not nished yet. We are building a new theory. How do you create a theory? Beauty is the guide. And this course will give a history of ideas for metabiology with plenty of examples. An idea is beautiful when it illuminates you, when it connects everything, when you ask yourself, Why didnt I see that before!, when in retrospect it seems obvious. AIT has two such ideas: the idea of looking at the size of a com- puter program as a complexity measure, and the idea of self-delimiting programs. Metabiology has two more beautiful ideas: the idea of or- ganisms as arbitrary programs with a dicult mathematical problem to solve, and the idea of mutations as arbitrary programs that operate on an organism to produce a mutated organism. Once you have these ideas, the rest is just uninspired routine work, lots of hard work, but thats all. In this course we shall discuss all four of these beautiful ideas, which were the key inspirations required for creating AIT and metabiology.
  • Introduction: Building a theory 15 Routine work is not enough, you need a spark from God. And mostly you need an instinct for mathematical beauty, for sensing an idea that can be developed, for the importance of an idea. That is, more than anything else, a question of aesthetics, of intuition, of instinct, of judge- ment, and it is highly subjective. I will try my best to explain why I believe in these ideas, but just as in artistic taste, there is no way to convince anyone. You either feel it somewhere deep in your soul or you dont. There is nothing more important than experiencing beauty; its a glimpse of transcendence, a glimpse of the divine, something that fewer and fewer people believe in nowadays. But without that we are mere machines. And I may have the beginnings of a mathematical theory of evolu- tion and biological creativity, but a mathematical theory of beauty is nowhere in sight. Incompleteness goes from being threatening to provoking creativity and being applied in order to keep our organisms evolving indenitely. Evo- lution stagnates in most models because the organisms achieve their goals. In my model the organisms are asked to achieve something that can never be fully achieved because of the incompleteness phe- nomenon. So my organisms keep getting better and better at what they are doing; they can never stop, because stopping would mean that they had a complete answer to a math problem to which incom- pleteness applies. Indeed, the three mathematical challenges that my organisms face, naming large integers, fast growing functions, and large transnite ordinals, are very concrete, tangible examples of the incom- pleteness phenomenon, which at rst seemed rather mysterious. Incompleteness is the reason that our organisms have to keep evolving forever, as they strive to become more and more complete, less and less incomplete. . . Incompleteness keeps our model of evolution from stagnating, it gives our organisms a mission, a raison detre. You have to go beyond incompleteness; incompleteness gives rise to creativity and evolution. Incompleteness sounds bad, but the other side of the coin is creativity and evolution, which are good. Now we give an outline of the course, consisting of Course Topics 19: 1. This introduction.
  • 16 Chaitin: Metabiology 2. The Search for the Perfect Language. (My talk at the Perimeter Insti- tute in Waterloo.) Umberto Eco, Lull, Leibniz, Cantor, Russell, Hilbert, Godel, Turing, AIT, . Kabbalah, Key to Universal Knowledge, God-like Power of Creation, the Golem! Mathematical theories are all incomplete (Godel, Turing, ), but pro- gramming languages are universal. Most concise programming lan- guages, self-delimiting programs. 3. Is the world built out of information? Is everything software? (My talk at the Technion in Haifa.) Physics of information: Quantum Information Theory; general relativ- ity and black holes, Beckenstein bound, holographic principle = every physical system contains a nite number of bits of information that grows as the surface area of the physical system, not as its volume (Lee Smolin, Three Roads to Quantum Gravity); derivation of Ein- steins eld equations for gravity from the thermodynamics of black holes (Ted Jacobson, Thermodynamics of Spacetime: The Einstein Equation of State). The rst attempt to construct a truly fundamental mathematical model for biology: von Neumann self-reproducing automata in a cellular automata world, a world in which magic works, a plastic world. See also: Edgar F. Codd, Cellular Automata. Konrad Zuse, Rechnen- der Raum (Calculating Space). Fred Hoyle, Ossians Ride. Freeman Dyson, The Sun, the Genome, and the Internet (1999), green technol- ogy. Craig Venter, genetic engineering, synthetic life. Technological applications: Seeds for houses, seeds for jet planes! Plant the seed in the earth just add water and sunlight. Universal con- structors, 3D printers = matter printers = printers for objects. Flexible manufacturing. Alchemy, Plastic reality. 4. Articial Life: Evolution Simulations. Low Level: Thomas Rays Tierra, Christoph Adamis Avida, Walter Fontanas ALchemy (Algorithmic Chemistry), Genetic algorithms. High Level: Exploratory concept-formation based on examining lots of examples in elementary number theory, experimental math with no
  • Introduction: Building a theory 17 proofs: Douglas Lenat (1984), Automated theory formation in math- ematics, AM. After a while these stop evolving. What about proofs instead of simulations? Seems impossiblesee the frontispiece quotes facing the title page, especially the one by Jacob Schwartzbut there is hope. See Course Topic 5 arguing that is provably a bridge from math to biology. 5. How Real Are Real Numbers? A History of . (My talk at WLU in Waterloo.) Course Topic 3 gives physical arguments against real numbers, and this course topic gives mathematical arguments against real numbers. These considerations about paradoxical real numbers will lead us straight to the halting probability . That is not how was actually discovered, but it is the best way of understanding . Its a Whig history: how it should have been, not how it actually was. The irreducible complexity real number proves that math is more biological than biology; this is the rst real bridge between math and biology. Biology is extremely complicated, and pure math is innitely complicated. The theme of as concentrated mathematical creativity is introduced here; this is important because is the organism that emerges through random evolution in Course Topic 8. Now lets get to work in earnest to build a mathematical theory of evolution and biological creativity. 6. Metabiology: Life as Evolving Software. Stephen Wolfram, NKS: the origin of life as the physical implementa- tion of a universal programming language; the Ubiquity of Univer- sality. Francois Jacob, bricolage, Nature is a cobbler, a tinkerer. Neil Shubin, Your Inner Fish. Stephen Gould, Wonderful Life, on the Cam- brian explosion of body designs. Murray Gell-Mann, frozen accidents. Ernst Haeckel, ontogeny recapitulates phylogeny. Evo-devo. Note that a small change in a computer program (one bit!) can com- pletely wreck it. But small changes can also make substantial improve- ments. This is a highly nonlinear eect, like the famous buttery eect
  • 18 Chaitin: Metabiology of chaos theory (see James Gleicks Chaos). Over the history of this planet, covering the entire surface of the earth, there is time to try many small changes. But not enough time according to the Intelli- gent Design book Signature in the Cell. In the real world this is still controversial, but in my toy model evolution provably works. A blog summarized one of my talks on metabiology like this: We are all random walks in program space! Thats the general idea; in Course Topics 7 and 8 we ll in the details of this new theory. 7. Creativity in Mathematics. We need to challenge our organisms into evolving. We need to keep them from stagnating. These problems can utilize an unlimited amount of mathematical creativity: Busy Beaver problem: Naming large integers: 1010 , 101010 . . . Naming fast-growing functions: N2 , 2N . . . Naming large transnite Cantor ordinals: , 2 , . . . 8. Creativity in Biology. Single mutating organism. Hill-climbing algo- rithm on a tness landscape. Hill-climbing random walks in software space. Evolution of mutating software. What is a mutation? Exhaus- tive search. Intelligent design. Cumulative evolution at random. as concentrated creativity, as an evolving organism. Randomness yields intelligence. We have a proof that evolution works, at least in this toy model; in fact, surprisingly it is nearly as fast as intelligent design, as deliberately choosing the mutations in the best possible order. But can we show that random evolution is slower than intelligent design? Otherwise the theory collapses onto a point, it cannot distinguish, it does not make useful distinctions. We also get evolution of hierarchical structure in non-universal programming languages. So we seem to have evolution at work in these toy models. But to what extent is this relevant to real biological systems? 9. Conclusion: On the plasticity of the world. Is the universe mental? Speculation where all this might possibly lead.
  • Chapter 2 The search for the perfect language I will tell how the story given in Umberto Ecos book The Search for the Perfect Language continues with modern work on logical and programming languages. Lecture given Monday, 21 September 2009, at the Perimeter In- stitute for Theoretical Physics in Waterloo, Canada.1 Today Im not going to talk much about . I will focus on that at Wilfrid Laurier University tomorrow. And if you want to hear a little bit about my current enthusiasm, which is what Im optimistically calling metabiology its a eld with a lovely name and almost no content at this time thats on Wednesday at the Institute for Quantum Computing. I thought it would be fun here at the Perimeter Institute to repeat a talk, to give a version of a talk, that I gave in Jerusalem a year ago. To understand the talk it helps to keep in mind that it was rst given in Jerusalem. Id like to give you a broad sweep of the history of mathematical logic. Im a math- ematician who likes physicists; some mathematicians dont like physicists. But I do. Before I became a mathematician I wanted to be a physicist. So Im going to talk about mathematics, and Id like to give you a broad overview, most denitely a non-standard view of some intellectual history. It 1 This lecture was published in Portuguese in Sao Paulo, Brazil, in the magazine Dicta & Contradicta, No. 4, 2009. See http://www.dicta.com.br/. 19
  • 20 Chaitin: Metabiology will be a talk about the history of work on the foundations of mathematics as seen from the perspective of the Middle Ages. So here goes. . . This talk = Umberto Eco + Hilbert, Godel, Turing. . . Outline at: http://www.cs.umaine.edu/~chaitin/hu.html There is a wonderful book by Umberto Eco called The Search for the Perfect Language, and I recommend it highly to all of you. In The Search for the Perfect Language you can see that Umberto Eco likes the Middle Ages I think he probably wishes we were still there. And this book talks about a dream that Eco believes played a fundamental role in European intellectual history, which is the search for the perfect language. What is the search for the perfect language? Nowadays a physicist would call this the search for a Theory of Everything (TOE), but in the terms in which it was formulated originally, it was the idea of nding, shall we say, the language of creation, the language before the Tower of Babel, the language that God used in creating the universe, the language whose structure directly expresses the structure of the world, the language in which concepts are expressed in their direct, original format. You can see that this idea is a little bit like the attempt to nd a foun- dational Theory of Everything in physics. The crucial point is that knowing this language would be like having a key to universal knowledge. If youre a theologian, it would bring you closer, very close, to Gods thoughts, which is dangerous. If youre a magician, it would give you magical powers. If youre a linguist, it would tell you the original, pure, uncorrupted language from which all languages descend. One can go on and on. . . This very fascinating book is about the quest to nd this language. If you nd it, youre opening a door to absolute knowledge, to God, to the ultimate nature of reality, to whatever. And there are a lot of interesting chapters in this intellectual history. One of them is Raymond Lull, around 1200, a Catalan. Raymond Lull 1200 He was a very interesting gentleman who had the idea of mechanically com- bining all possible concepts to get new knowledge. So you would have a wheel with dierent concepts on it, and another wheel with other concepts on it, and you would rotate them to get all possible combinations. This would be
  • The search for the perfect language 21 a systematic way to discover new concepts and new truths. And if you re- member Swifts Gullivers Travels, there Swift makes fun of an idea like this, in one of the parts of the book that is not for children but denitely only for adults. Lets leave Lull and go on to Leibniz. In The Search for the Perfect Language there is an entire chapter on Leibniz. Leibniz is a transitional gure in the search for the perfect language. Leibniz is wonderful because he is universal. He knows all about Kabbalah, Christian Kabbalah and Jewish Kabbalah, and all kinds of hermetic and esoteric doctrines, and he knows all about alchemy, he actually ghost-authored a book on alchemy. Leibniz knows about all these things, and he knows about ancient philosophy, he knows about scholastic philosophy, and he also knows about what was then called mechanical philosophy, which was the beginning of modern science. And Leibniz sees good in all of this. And he formulates a version of the search for the perfect language, which is rmly grounded in the magical, theological original idea, but which is also t for consumption nowadays, that is, acceptable to modern ears, to contem- porary scientists. This is a universal language he called the characteristica universalis that was supposed to come with a crucial calculus ratiocinator. Leibniz: characteristica universalis, calculus ratiocinator The idea, the goal, is that you would reduce reasoning to calculation, to computation, because the most certain thing is that 2 + 5 = 7. In other words, the way Leibniz put it, perhaps in one of his letters, is that if two people have an intellectual dispute, instead of dueling they could just sit down and say, Gentlemen, let us compute!, and get the correct answer and nd out who was right. So this is Leibnizs version of the search for the perfect language. How far did he get with this? Well, Leibniz is a person who gets bored easily, and ies like a buttery from eld to eld, throwing out fundamental ideas, rarely taking the trouble to develop them fully. One case of the characteristica universalis that Leibniz did develop is called the calculus. This is one case where Leibniz worked out his ideas for the perfect language in beautiful detail. Leibnizs version of the calculus diers from Newtons precisely because it is part of Leibnizs project for the characteristica universalis. Christian Huygens hated the calculus.
  • 22 Chaitin: Metabiology Christian Huygens taught Leibniz mathematics in Paris at a relatively late age, when Leibniz was in his twenties. Most mathematicians start very, very young. And Christian Huygens hated Leibnizs calculus because he said that it was mechanical, it was brainless: Any fool can just calculate the answer by following the rules, without understanding what he or she is doing. Huygens preferred the old, synthetic geometry proofs where you have to be creative and come up with a diagram and some particular reason for something to be true. Leibniz wanted a general method. He wanted to get the formalism, the notation, right, and have a mechanical way to get the answer. Huygens didnt like this, but that was precisely the point. This was precisely what Leibniz was looking for, for everything! The idea was that if you get absolute truth, if you have found the truth, it should mechanically enable you to determine whats going on, without creativity. This is good, this is not bad. This is also precisely how Leibnizs version of the calculus diered from Newtons. Leibniz saw clearly the importance of having a formalism that led you automatically to the answer. Lets now take a big jump, to David Hilbert, about a century ago. . . No, rst I want to tell you about an important attempt to nd the perfect language: Cantors theory of innite sets. Cantor: Innite Sets This late 19th century theory is interesting because its rmly in the Middle Ages and also, in a way, the inspiration for all of 20th century mathematics. This theory of innite sets was actually theology. This is mathematical theology. Normally you dont mention that fact. To be a eld of mathe- matics, the price of admission is you throw out all the philosophy, and you just end up with something technical. So all the theology has been thrown out. But Cantors goal was to understand God. God is transcendent. The theory of innite sets has this hierarchy of bigger and bigger innities, the alephs, the s. You have 0, 1, the innity of integers, of real numbers, and you keep going. Each one of these is the set of all subsets of the previous one. And very far out you get mind-boggling innities like ; this is the rst innity after 0, 1, 2, 3, 4 . . .
  • The search for the perfect language 23 Then you can continue with + 1, + 2, + 3 . . . 2 + 1, 2 + 2, 2 + 3 . . . These so-called ordinal numbers are subscripts for the s, which are cardi- nalities. Lets go farther: 2 , , . . . And theres an ordinal called epsilon-nought 0 = ... which is the smallest solution of the equation x = x . And the corresponding cardinal 0 is pretty big! You know, God is very far o, since God is innite and transcendent. We can try to go in His direction. But were never going to get there, because after every cardinal, theres a bigger one, the cardinality of the set of all subsets. And after any innite sequence of cardinals that you get, you just take the union of all of that, and you get a bigger cardinal than is in the sequence. So this thing is inherently open-ended. And contradictory, by the way! Theres only one problem. This is absolutely wonderful, breath-taking stu. The only problem is that its contradictory. The problem is very simple. If you take the universal set, the set of everything, and you consider the set of all its subsets, by Cantors diago- nal argument this should have a bigger cardinality, but how can you have anything bigger than the set of everything? This is the paradox that Bertrand Russell discovered. Russell looked at this and asked why do you get this bad result. And if you look at the Cantor diagonal argument proof that the set of all subsets of everything is bigger than everything, it involves the set of all sets that are not members of themselves, {x : x x},
  • 24 Chaitin: Metabiology which can neither be in itself nor not be in itself. This is called the Russell paradox. Cantor was aware of the fact that this happens, but Cantor wasnt both- ered by these contradictions, because he was doing theology. Were nite but God is innite, and its paradoxical for a nite being to try to comprehend a transcendent, innite being, so paradoxes are okay. But the math community is not very happy with a theory which leads to contradictions. However, these ideas are so wonderful, that what the math community has done is forget about all this theology and philosophy and try to sweep the contradictions under the rug. There is an expurgated version of all this called Zermelo-Fraenkel set theory, with the axiom of choice, usually: ZFC. This is a formal axiomatic theory which you develop using rst-order logic, and it is an expurgated version of Cantors theory believed not to contain any paradoxes. Anyway, Bertrand Russell was inspired by all of this to attempt a general critique of mathematical reasoning, and to nd a lot of contradictions, a lot of mathematical arguments that lead to contradictions. Bertrand Russell: mathematics is full of contradictions. I already told you about his most famous one, the Russell paradox. Russell was an atheist who was searching for the absolute, who believed in absolute truth. And he loved mathematics and wanted mathematics to be perfect. Russell went around telling people about these contradictions in order to try to get them xed. Besides the paradox that theres no biggest cardinal and that the set of subsets of everything is bigger than everything, theres also a problem with the ordinal numbers thats called the Burali-Forti paradox, namely that the set of all the ordinals is an ordinal thats bigger than all the ordinals. This works because each ordinal can be dened as the set of all the ordinals that are smaller than it is. (Then an ordinal is less than another ordinal if and only if it is contained in it.) Russell is going around telling people that reason leads to contradictions. So David Hilbert about a century ago proposes a program to put mathematics on a rm foundation. And basically what Hilbert proposes is the idea of a completely formal axiomatic theory, which is a modern version of Leibnizs characteristica universalis and calculus ratiocinator: David Hilbert: mathematics is a formal axiomatic theory.
  • The search for the perfect language 25 This is the idea of making mathematics totally objective, of removing all the subjective elements. So in such a formal axiomatic theory you would have a nite number of axioms, axioms that are not written in an ambiguous natural language. Instead you use a precise articial language with a simple, regular articial grammar. You use mathematical logic, not informal reasoning, and you specify the rules of the game completely precisely. It should be mechanical to decide whether a proof is correct. Hilbert was a conservative. He believed that mathematics gives abso- lute truth, which is an idea from the Middle Ages. You can see the Middle Ages whenever you mention absolute truth. Nevertheless, modern mathe- maticians remain enamored with absolute truth. As Godel said, we pure mathematicians are the last holdout of the Middle Ages. We still believe in the Platonic world of ideas, at least mathematical ideas, when everyone else, including philosophers, now laughs at this notion. But pure mathemati- cians live in the Platonic world of ideas, even though everyone else stopped believing in this a long time ago. So math gives absolute truth, said Hilbert. Every mathematician some- where deep inside believes this. Then there ought to exist a nite set of axioms, and a precise set of rules for deduction, for inference, such that all of mathematical truth is a consequence of these axioms. You see, if mathemat- ical truth is black or white, and purely objective, then if you ll in all the steps in a proof and carefully use an articial language to avoid ambiguity, you should be able to have a nite set of axioms we can all agree on, that in principle enable you to deduce all of mathematical truth. This is just the notion that mathematics provides absolute certainty; Hilbert is analyzing what this means. What Hilbert says is that the traditional view that mathematics provides absolute certainty, that in the Platonic world of pure mathematics everything is black or white, means that there should be a single formal axiomatic theory for all of math. That was a very important idea of his. An important consequence of this idea goes back to the Middle Ages. This perfect language for mathematics, which is what Hilbert was looking for, would in fact give a key to absolute knowledge, because in principle you could mechanically deduce all the theorems from the axioms, simply by running through the tree of all possible proofs. You start with the axioms, then you apply the rules of inference once, and get all the theorems that have one-step proofs, you apply them two times, and you get all the theorems that
  • 26 Chaitin: Metabiology have two-step proofs, and like that, totally mechanically, you would get all of mathematical truth, by systematically traversing the tree of all possible proofs. This would not put all mathematicians out of work, not at all. In practice this process would take an outrageous amount of time to get to interesting results, and all the interesting theorems would be overwhelmed by uninter- esting theorems, such as the fact that 1 + 1 = 2 and other trivialities. It would be hard to nd the interesting theorems and to separate the wheat from the cha. But in principle this would give you all mathematical truths. You wouldnt actually do it, but it would show that math gives absolute certainty. By the way, it was important to make all mathematicians agree on the choice of formal axiomatic theory, and you would use metamathematics to try to convince everyone that this formal axiomatic theory avoids all the paradoxes that Bertrand Russell had noticed and contains no contradictions. Okay, so this was the idea of putting mathematics on a rm foundation and removing all doubts. This was Hilberts idea, about a century ago, and metamathematics studies a formal axiomatic theory from the outside, and notice that this is a door to absolute truth, following the notion of the perfect language. So what happens with this program, with this proposal of Hilberts? Well, theres some good news and some bad news. Some of the good news I already mentioned: The thing that comes the closest to what Hilbert asked for is Zermelo-Fraenkel set theory, and it is a beautiful axiomatic theory. I want to mention some of the milestones in the development of this theory. One of them is the von Neumann integers, so let me tell you about that. Remember that Spinoza has a philosophical system in which the world is built out of only one substance, and that substance is God, thats all there is. Zermelo-Fraenkel set theory is similar. Everything is sets, and every set is built out of the empty set. Thats all there is: the empty set, and sets built starting with the empty set. So zero is the empty set, thats the rst von Neumann integer, and in general n + 1 is dened to be the set of all integers less than or equal to n: von Neumann integers: 0 = {}, n + 1 = {0, 1, 2, . . . , n}. So if you write this out in full, removing all the abbreviations, all you have are curly braces, you have set formation starting with no content, and the
  • The search for the perfect language 27 full notation for n grows exponentially in n, if you write it all out, because everything up to that point is repeated in the next number. In spite of this exponential growth, this is a beautiful conceptual scheme. Then you can dene rational numbers as pairs of these integers, you can dene real numbers as limit sequences of rationals, and you get all of mathematics, starting just with the empty set. So its a lovely piece of ontology. Heres all of mathematical creation just built out of the empty set. And other people who worked on this are of course Fraenkel and Zermelo, because it is called Zermelo-Fraenkel set theory, and an approximate notion of what they did was to try to avoid sets that are too big. The universal set is too big, it gets you into trouble. Not every property determines a set. So this is a formal theory that most mathematicians believe enables you to carry out all the arguments that normally appear in mathematics maybe if you dont include category theory, which is very dicult to formalize, and even more paradoxical than set theory, from what I hear. Okay, so thats some of the positive work on Hilberts program. Now some of the negative work on Hilberts program Id like to tell you about it, youve all heard of it is of course Godel in 1931 and Turing in 1936. Godel, 1931 Turing, 1936 What they show is that you cant have a perfect language for mathematics, you cannot have a formal axiomatic theory like Hilbert wanted for all of mathematics, because of incompleteness, because no such system will include all of mathematical truth, it will always leave out truths, it will always be incomplete. And this is Godels incompleteness theorem of 1931, and Godels original proof is very strange. Its basically the paradox of this statement is false, This statement is false! which is a paradox of course because it can be neither true nor false. If its false that its false, then its true, and if its true that its false, then its false. Thats just a paradox. But what Godel does is say this statement is unprovable. This statement is unprovable! So if the statement says of itself its unprovable, there are two possibilities: its provable, or it isnt.
  • 28 Chaitin: Metabiology If its provable, then were proving something thats false, because it says its unprovable. So we hope thats not the case; by hypothesis, well eliminate that possibility. If we prove things that are false, we have a formal axiomatic theory that were not interested in, because it proves false things. The only possibility left is that its unprovable. But if its unprovable then its true, because it asserts its unprovable, therefore theres a hole. We havent captured all of mathematical truth in our theory. This proof of incompleteness shocks a lot of people, but my personal reaction to it is, okay, its correct, but I dont like it. A better proof of incompleteness, a deeper proof, comes from Turing in 1936. He derives incompleteness from a more fundamental phenomenon, which is uncomputability, the discovery that mathematics is full of stu that cant be calculated, of things you can dene, but which you cannot calculate, because theres no algorithm. Uncomputability Incompleteness And in particular, the uncomputable thing that he discovers is the halt- ing problem, a very simple question: Does a computer program thats self-contained halt or does it go on forever? There is no algorithm to answer this in every individual case, therefore there is no formal axiomatic theory that enables you to always prove in individual cases what the answer is. Why not? Because if there were a formal axiomatic theory thats complete for the halting problem, that would give you a mechanical procedure for deciding, by running through the tree of all possible proofs, until you nd a proof that an individual program youre interested in halts, or you nd a proof that it doesnt. But thats impossible because this is not a computable function. So Turings insight in 1936 is that incompleteness, that Godel found in 1931, for any formal axiomatic theory, comes from a deeper phenomenon, which is uncomputability. Incompleteness is an immediate corollary of un- computability, a concept which does not appear in Godels 1931 paper. But Turings paper has both good and bad aspects. Theres a negative aspect of his 1936 paper, which Ive just told you about, but theres also a positive aspect. You get another proof, a deeper proof of incompleteness, but you also get a kind of completeness. You nd a perfect language. There is no perfect language for mathematical reasoning. Godel showed that in 1931, and Turing showed it again in 1936. But what Turing also
  • The search for the perfect language 29 showed in 1936 is that there are perfect languages, not for mathematical reasoning, but for computation, for specifying algorithms. What Turing discovers in 1936 is that theres a kind of completeness called universality and that there are universal Turing machines and universal programming languages. Universal Turing Machines / Programming Languages What universal means, what a universal programming language or a uni- versal Turing machine is, is a language in which every possible algorithm can be written. So on the one hand, Turing shows us in a deeper way that any language for mathematical reasoning has to be incomplete, but on the other hand, he shows us that languages for computation can be universal, which is just another name, a synonym, for completeness. There are perfect languages for computation, for writing algorithms, even though there arent any perfect languages for mathematical reasoning. This is the positive side, this is the completeness side, of Turings 1936 paper. Now, what Ive spent most of my professional life on, is a subject I call algorithmic information theory Algorithmic Information Theory (AIT) that derives incompleteness from uncomputability by taking advantage of a deeper phenomenon, by considering an extreme form of uncomputability, which is called algorithmic randomness or algorithmic irreducibility. AIT: algorithmic randomness, algorithmic irreducibility Theres a perfect language again, and theres also a negative side, the halt- ing probability , whose bits are algorithmically random, algorithmically irreducible mathematical truths. = .010010111 . . . This is a place in pure mathematics where theres no structure. If you want to know the bits of the numerical value of the halting probability, this is a well-dened mathematical question, and in the world of mathematics all truths are necessary truths, but these look like accidental, contingent truths. They look random, they have irreducible complexity.
  • 30 Chaitin: Metabiology This is a maximal case of uncomputability, this is a place in pure mathe- matics where theres absolutely no structure at all. Although it is true that you can in a few cases actually know some of the rst bits. . . There are actually an innite number of halting probabilities depending on your choice of programming language. After you choose a language, then you ask what is the probability that a program generated by coin tossing will eventually halt. And that gives you a dierent halting probability. The numerical value will be dierent; the paradoxical properties are the same. Okay, there are cases for which you can get a few of the rst bits. For example, if starts with 1s in binary or 9s in decimal, you can know those bits or digits, if is .11111. . . base two or .99999. . . base ten. So you can get a nite number of bits, perhaps, of the numerical value, but if you have an N- bit formal axiomatic theory, then you cant get more than N bits of . Thats sort of the general result. Its irreducible logically and computationally. Its irreducible mathematical information. Its a perfect simulation in pure math, where all truths are necessary, of contingent, accidental, maximal entropy truths. So thats the bad news from AIT. But just like in Turings 1936 work, there is a positive side. On the one hand we have maximal uncomputabil- ity, maximal entropy, total lack of structure, of any redundancy, in an information-theoretic sense, but theres also good news. AIT, the theory of program-size complexity, the theory where is the crown jewel, goes further than Turing, and picks out from Turings universal Turing machines, from Turings universal languages, maximally expressive programming languages. Because those are the ones that you have to use to develop this theory where you get to . AIT has the notion of a maximally expressive programming language in which programs are maximally compact, and deals with a very basic complex- ity concept which is the size of the smallest program to calculate something: H(x) is the size in bits of the smallest program to calculate x. And we now have a better notion of perfection. The perfect languages that Turing found, the universal programming languages, are not all equally good. We now concentrate on a subset, the ones that enable us to write the most concise programs. These are the most expressive languages, the ones with the smallest programs. Now let me tell you, this denition of complexity is a dry, technical way of expressing this idea in modern terms. But let me put this into Medieval
  • The search for the perfect language 31 terminology, which is much more colorful. The notion of program-size com- plexity which by the way has many dierent names: algorithmic complex- ity, Kolmogorov complexity, algorithmic information content in Medieval terms, what were asking is, how many yes/no decisions did God have to make to create something?, which is obviously a rather basic question to ask. That is, if you consider that God is calculating the universe. Im giving you a Medieval perspective on these modern developments. Theology is the fundamental physics, its the theoretical physics of the Middle Ages. I have a lot of time left Ive been racing through this material so maybe I should explain in more detail how AIT contributes to the quest for the perfect language. The notion of universal Turing machine that is used in AIT is Turings very basic idea of a exible machine. Its exible hardware, which we call soft- ware. In a way, Turing in 1936 creates the computer industry and computer technology. Thats a tremendous benet of a paper that mathematically sounds at rst rather negative, since it talks about things that cannot be calculated, that cannot be proved. But on the other hand theres a very pos- itive aspect I stated it in theoretical terms which is that programming languages can be complete, can be universal, even though formal axiomatic theories cannot be complete. Okay, so you get this technology, theres this notion of a exible machine, this notion of software, which emerges in this paper. Von Neumann, the same von Neumann who invented the von Neumann integers, credited all of this to Turing. At least Turing is responsible for the concept; the hardware implementation is another matter. Now, AIT, where you talk about program-size complexity, the size of the smallest program, how many yes/no decisions God has to make to calcu- late something, to create something, picks out a particular class of universal Turing machines U. What are the universal computers U like that you use to dene program- size complexity and talk about ? Well, a universal computer U has the property that for any other computer C and its program p, your universal computer U will calculate the same result if you give it the original program p for C concatenated to a prex C which depends only on the computer C that you want to simulate. C tells U which computer to simulate. In symbols, U(C p) = C(p).
  • 32 Chaitin: Metabiology In other words, C p is the concatenation of two pieces of information. Its a binary string. You take the original program p, which is also a binary string, and in front of it you put a prex that tells you which computer to simulate. Which means that these programs C p for U are only a xed number of bits larger than the programs p for any individual machine C. These U are the universal Turing machines that you use in AIT. These are the most expressive languages. These are the languages with maximal expressive power. These are the languages in which programs are as concise as possible. This is how you dene program-size complexity. God will natu- rally use the most perfect, most powerful programming languages, when he creates the world, to build everything. I should point out that Turings original universality concept was not careful about counting bits; it didnt really care about the size of programs. All a universal machine U had to do was to be able to simulate any other machine C, but one did not study the size of the program for U as a function of the size of the program for C. Here we are careful not to waste bits. AIT is concerned with particularly ecient ways for U to be universal. The original notion of universality in Turing was not this demanding. The fact that you can just add a xed number of bits to a program for C to get one for U is not completely trivial. Let me tell you why. After you put C and p together, you have to know where the prex ends and the program that is being simulated begins. There are many ways to do this. A very simple way to make the prex C self-delimiting is to have it be a sequence of 0s followed by a 1: C = 0k 1. And the number k of 0s tells us which machine C to simulate. Thats a very wasteful way to indicate this. The prex C is actually an interpreter for the programming language C. AITs universal languages U have the property that you give U an interpreter plus the program p in this other language C, and U will run the interpreter to see what p does. If you think of this interpreter C as an arbitrary string of bits, one way to make it self-delimiting is to just double all the bits. 0 goes to 00, 1 goes to 11, and you put a pair of unequal bits 01 as punctuation at the end:
  • The search for the perfect language 33 Arbitrary C: 0 00, 1 11, 01 at the end. This is a better way to have a self-delimiting prex that you can concatenate with p. It only doubles the size, the 0k 1 trick increases the size exponentially. And there are more ecient ways to make the prex self-delimiting. For example, you can put the size of the prex in front of the prex. But its sort of like Russian dolls, because if you put the size |C| of C in front of C, |C| also has to be self-delimiting: U(. . . ||C|| |C| C p) = C(p). Anyway, picking U this way is the key idea in the original 1960s version of AIT that Solomono, Kolmogorov and I independently proposed. But ten years later I realized that this is not the right approach. You actually want the whole program C p for U to be self-delimiting, not just the prex C. You want the whole thing to be self-delimiting to get the right theory of program-size complexity. Let me compare the 1960s version of AIT and the 1970s version of AIT. Let me compare these two dierent theories of program-size complexity. In the 1960s version, an N-bit string will in general need an N-bit pro- gram, if its irreducible, and most strings are algorithmically irreducible. Most N-bit strings need an N-bit program. These are the irreducible strings, the ones that have no pattern, no structure. Most N-bit strings need an N- bit program, because there arent enough smaller programs. But in the 1970s version of AIT, you go from N bits to N + log2 N bits, because you want to make the programs self-delimiting. An N-bit string will usually need an N + log2 N bit program: Most N-bit strings AIT1960: N bits of complexity, AIT1970: N + log2 N bits of complexity. Actually, in AIT1970 its N plus H(N), which is the size of the smallest self-delimiting program to calculate N, thats exactly what that logarithmic term is. In other words, in the 1970s version of AIT, the size of the smallest program for calculating an N-bit string is usually N bits plus the size in bits of the smallest self-delimiting program to calculate N, which is roughly log N + log log N + log log log N + . . .
  • 34 Chaitin: Metabiology bits long. Thats the Russian dolls aspect of this. The 1970s version of AIT, which takes the idea of being self-delimiting from the prex and applies it to the whole program, gives us even better perfect languages. AIT evolved in two stages. First we concentrate on those U with U(C p) = C(p) with C self-delimiting, and then we insist that the whole thing C p has also got to be self-delimiting. And when you do that, you get important new results, such as the sub-additivity of program-size complexity, H(x, y) H(x) + H(y), which is not the case if you dont make everything self-delimiting. This just says that you can concatenate the smallest program for calculating x and the smallest program for calculating y to get a program for calculating x and y. And you cant even dene the halting probability in AIT1960. If you allow all N-bit strings to be programs, then you cannot dene the halting probability in a natural way, because the sum for dening the probability that a program will halt = p halts 2(size in bits of p) diverges to innity instead of being between zero and one. This is the key technical point in AIT. I want the halting probability to be nite. The normal way of thinking about programs is that there are 2N N-bit programs, and the natural way of dening the halting probability is that every N-bit program that halts contributes 1/2N to the halting probability. The only problem is that for any xed size N there are roughly order of 2N programs that halt, so if you sum over all possible sizes, you get innity, which is no good. In order to get the halting probability to be between zero and one 0 < = p halts 2(size in bits of p) < 1 you have to be sure that the total probability summed over all programs p is less than or equal to one. This happens automatically if we force p to be self-delimiting. How can we do this? Easy! Pretend that you are the
  • The search for the perfect language 35 universal computer U. As you read the program bit by bit, you have to be able to decide by yourself where the program ends, without any special punctuation, such as a blank, at the end of the program. This implies that no extension of a valid program is a valid program, and that the set of valid programs is whats called a prex-free set. Then the fact that the sum that denes must be between zero and one, is just a special case of whats called the Kraft inequality in Shannon information theory. But this technical machinery isnt necessary. That 0 < < 1 follows immediately from the fact that as you read the program bit by bit you are forced to decide where to stop without seeing any special punctuation. In other words, in AIT1960 we were actually using a three-symbol alphabet for programs: 0, 1 and blank. The blank told us where a program ends. But thats a symbol that youre wasting, because you use it very little. As you all know, if you have a three-symbol alphabet, then the right way to use it is to use each symbol roughly one-third of the time. So if you really use only 0s and 1s, then you have to force the Turing machine to decide by itself where the program ends. You dont put a blank at the end to indicate that. So programs go from N bits in size to N +log2 N bits, because youve got to indicate in each program how big it is. On the other hand, you can just take subroutines and concatenate them to make a bigger program, so program- size complexity becomes sub-additive. You run the universal machine U to calculate the rst object x, and then you run it again to calculate the second object y, and then youve got x and y, and so H(x, y) H(x) + H(y). These self-delimiting binary languages are the ones that the study of program-size complexity has led us to discriminate as the ideal languages, the most perfect languages. We got to them in two stages, AIT1960 and AIT1970. These are languages for computation, for expressing algorithms, not for mathematical reasoning. They are universal programming languages that are maximally expressive, maximally concise. We already knew how to do that in the 1960s, but in the 1970s we realized that programs should be self-delimiting, which made it possible to dene the halting probability . Okay, so thats the story, and now maybe I should summarize all of this, this saga of the quest for the perfect language. As I said, the search for the perfect language has some negative conclusions and some positive conclu- sions.
  • 36 Chaitin: Metabiology Hilbert wanted to nd a perfect language giving all of mathematical truth, all mathematical knowledge, he wanted a formal axiomatic theory for all of mathematics. This was supposed to be a Theory of Everything for the world of pure math. And this cannot succeed, because we know that every formal axiomatic theory is incomplete, as shown by Godel, by Turing, and by my halting probability . Instead of nding a perfect language, a perfect for- mal axiomatic theory, we found incompleteness, uncomputability, and even algorithmic irreducibility and algorithmic randomness. So thats the negative side of this story, which is fascinating from an epistemological point of view, because we found limits to what we can know, we found limits of formal reasoning. Now interestingly enough, the mathematical community couldnt care less. They still want absolute truth! They still believe in absolute truth, and that mathematics gives absolute truth. And if you want a proof of this, just go to the December 2008 issue of the Notices of the American Mathematical Society. Thats a special issue of the Notices devoted to formal proof. The technology has been developed to the point where they can run real mathematics, real proofs, through proof-checkers, and get them checked. A mathematician writes the proof out in a formal language, and lls in the missing steps and makes corrections until the proof-checker can understand the whole thing and verify that it is correct. And these proof-checkers are getting smarter and smarter, so that more and more of the details can be left out. As the technology improves, the job of formalizing a proof becomes easier and easier. The formal-proof extremists are saying that in the future all mathematics will have to be written out formally and veried by proof-checkers. The engineering has been worked out to the point that you can formally prove real mathematical results and run them through proof-checkers for verication. For example, this has been done with the proof of the four-color conjecture. It was written out as a formal proof that was run through a proof-checker. And the position of these extremists is that in the future all mathematics will have to be written out in a formal language, and you will have to get it checked before submitting a paper to a human referee, who will then only have to decide if the proof is worth publishing, not whether the proof is correct. And they want a repository of all mathematical knowledge, which would be a database of checked formal proofs of theorems. This is a substantial community, and to learn more, go to the December
  • The search for the perfect language 37 2008 AMS Notices, which is available on the web for free in the AMS website. This is being worked on by a sizeable community, and the Notices devoted a special issue to it, which means that mathematicians still believe in absolute truth. Im not disparaging this extremely interesting work, but I am saying that theres a wonderful intellectual tension between it and the incompleteness re- sults that Ive discussed in this talk. Theres a wonderful intellectual tension between incompleteness and the fact that people still believe in formal proof and absolute truth. People still want to go ahead and carry out Hilberts program and actually formalize everything, just as if Godel and Turing had never happened! I think this is an extremely interesting and, at least for me, a quite unexpected development. These were the negative conclusions from this saga. Now I want to wrap this talk up by summarizing the positive conclusions. There are perfect languages, for computing, not for reasoning. Theyre computer programming languages. And we have universal Turing machines and universal programming languages, and although languages for reason- ing cannot be complete, these universal programming languages are com- plete. Furthermore, AIT has picked out the most expressive programming languages, the ones that are particularly good to use for a theory of program- size complexity. So there is a substantial practical spino. Furthermore, since Ive worked most of my professional career on AIT, I view AIT as a substantial contri- bution to the search for the perfect language, because it gives us a measure of expressive power, and of conceptual complexity and the complexity of ideas. Remember, I said that from the perspective of the Middle Ages, thats how many yes/no decisions God had to make to create something, which obviously He will do in an optimum manner.2 From the theoretical side, however, this quest was disappointing due to Godel incompleteness and because there is no Theory of Everything for pure math. Provably there is no TOE for pure math. In fact, if you look at the bits of the halting probability , they show that pure mathematics contains innite irreducible complexity, and in this precise sense is more like biology, the domain of the complex, than like theoretical physics, where there is still 2 Note that program-size complexity = size of smallest name for something.
  • 38 Chaitin: Metabiology hope of nding a simple, elegant TOE.3 So this is the negative side of the story, unless youre a biologist. The positive side is we get this marvelous programming technology. So this dream, the search for the perfect language and for absolute knowledge, ended in the bowels of a computer, it ended in a Golem. In fact, let me end with a Medieval perspective on this. How would all this look to someone from the Middle Ages? This quest, the search for the perfect language, was an attempt to obtain magical, God-like powers. Lets bring someone from the 1200s here and show them a notebook computer. You have this dead machine, its a machine, its a physical object, and when you put software into it, all of a sudden it comes to life! So from the perspective of the Middle Ages, I would say that the perfect languages that weve found have given us some magical, God-like powers, which is that we can breath life into some inanimate matter. Observe that hardware is analogous to the body, and software is analogous to the soul, and when you put software into a computer, this inanimate object comes to life and creates virtual worlds. So from the perspective of somebody from the year 1200, the search for the perfect language has been successful and has given us some magical, God-like abilities, except that we take them entirely for granted. Thanks very much!4 3 Incompleteness can be considered good rather than bad: It shows that mathematics is creative, not mechanical. 4 Twenty minutes of questions and discussion followed. These have not been transcribed, but are available via digital streaming video at http://pirsa.org/09090007/.
  • Chapter 3 Is the world built out of information? Is everything software? From Chaitin, Costa, Doria, After Godel, in preparation. Lecture, the Technion, Haifa, Thursday, 10 June 2010. Now for some even weirder stu! Lets return to The Thirteenth Floor and to the ideas that we briey referred to in the introductory section of this chapter. Lets now turn to ontology: What is the world built out of, made out of? Fundamental physics is currently in the doldrums. There is no pressing unexpected, new experimental data or if there is, we cant see that it is! So we are witnessing a return to pre-Socratic philosophy with its em- phasis on ontology rather than epistemology. We are witnessing a return to metaphysics. Metaphysics may be dead in contemporary philosophy, but amazingly enough it is alive and well in contemporary fundamental physics and cosmology. There are serious problems with the traditional view that the world is a space-time continuum. Quantum eld theory and general relativity con- tradict each other. The notion of space-time breaks down at very small distances, because extremely massive quantum uctuations (virtual parti- cle/antiparticle pairs) should provoke black holes and space-time should be torn apart, which doesnt actually happen. Here are two other examples of problems with the continuum, with very 39
  • 40 Chaitin: Metabiology small distances: the innite self-energy of a point electron in classical Maxwell electro- dynamics, and in quantum eld theory, renormalization, which Dirac never ac- cepted. And here is an example of renormalization: the innite bare charge of the electron which is shielded by vacuum polarization via virtual pair formation and annihilation, so that far from an electron it only seems to have nite charge. This is analogous to the behavior of water, which is a highly polarized molecule forming micro-clusters that shield charge, with many of the highly positive hydrogen-ends of H2O near the highly negative oxygen-ends of these water molecules. In response to these problems with the continuum, some of us feel that the traditional Pythagorian ontology: God is a mathematician, the world is built out of mathematics, should be changed to this more modern Neo-Pythagorian ontology: God is a programmer, the world is built out of software. In other words, all is algorithm! There is an emerging school, a new viewpoint named digital philosophy. Here are some key people and key works in this new school of thought: Ed- ward Fredkin, http://www.digitalphilosophy.org, Stephen Wolfram, A New Kind of Science, Konrad Zuse, Rechnender Raum (Calculating Space), John von Neumann, Theory of Self-Reproducing Automata, and Chaitin, Meta Math!.1 These may be regarded as works on metaphysics, on possible digital worlds. However there have in fact been parallel developments in the world of physics itself. 1 Lesser known but important works on digital philosophy: Arthur Burks, Essays on Cellular Automata, Edgar Codd, Cellular Automata.
  • Is the world built out of information? Is everything software? 41 Quantum information theory builds the world out of qubits, not matter. And phenomenological quantum gravity and the theory of the entropy of black holes suggests that any physical system contains only a nite number of bits of information that grows, amazingly enough, as the surface area of the physical system, not as its volume hence the name holographic principle. For more on the entropy of black holes, the Bekenstein bound, and the holographic principle, see Lee Smolin, Three Roads to Quantum Gravity. One of the key ideas that has emerged from this research on possible digital worlds is to transform the universal Turing machine, a machine capable of running any algorithm, into the universal constructor, a ma- chine capable of building anything: Universal Turing Machine Universal Constructor. And this leads to the idea of an information economy: worlds in which everything is software, worlds in which everything is information and you can construct anything if you have a program to calculate it. This is like magic in the Middle Ages. You can bring something into being by invoking its true name. Nothing is hardware, everything is software!2 A more modern version of this everything-is-information view is presented in two green-technology books by Freeman Dyson: The Sun, the Genome and the Internet, and A Many-Colored Glass. He envisions seeds to grow houses, seeds to grow airplanes, seeds to grow factories, and imagines children using genetic engineering to design and grow new kinds of owers! All you need is water, sun and soil, plus the right seeds! From an abstract, theoretical mathematical point of view, the key concept here is an old friend from Chapter 2: H(x) = the size in bits of the smallest program to compute x. H(x) is also = to the minimum amount of algorithmic information needed to build/construct x, = in Medieval language the number of yes/no decisions God had to make to create x, = in biological terms, roughly the amount of DNA needed for growing x. It requires the self-delimiting programs of Chapter 2 for the following intuitively necessary condition to hold: H(x, y) H(x) + H(y) + c. 2 On magic in the Middle Ages, see Umberto Eco, The Search for the Perfect Language, and Allison Coudert, Leibniz and the Kabbalah.
  • 42 Chaitin: Metabiology This says that algorithmic information is sub-additive: If it takes H(x) bits of information to build x and H(y) bits of information to build y, then the sum of that suces to build both x and y. Furthermore, the mutual information, the information in common, has this important property: H(x) + H(y) H(x, y) = H(x) H(x|y ) + O(1), H(y) H(y|x ) + O(1). Here H(x|y) = the size in bits of the smallest program to compute x from y. This triple equality tells us that the extent to which it is better to build x and y together rather than separately (the bits of subroutines that are shared, the amount of software that is shared) is also equal to the extent that knowing a minimum-size program y for y helps us to know x and to the extent to which knowing a minimum-size program x for x helps us to know y. (This triple equality is an idealization; it holds only in the limit of extremely large compute times for x and y.) These results about algorithmic information/complexity H are a kind of economic meta-theory for the information economy, which is the asymp- totic limit, perhaps, of our current economy in which material resources (petroleum, uranium, gold) are still important, not just technological and scientic know-how. But as astrophysicist Fred Hoyle points out in his science ction novel Ossians Ride, the availability of unlimited amounts of energy, say from nu- clear fusion reactors, would make it possible to use giant mass spectrometers to extract gold and other chemical elements directly from sea water and soil. Material resources would no longer be that important. If we had unlimited energy, all that would matter would be know-how, information, knowing how to build things. And so we nally end up with the idea of a printer for objects, a more plebeian term for a universal constructor. There are already commercial versions of such devices. They are called 3D printers and are used for rapid prototyping and digital fabrication. They are not yet universal constructors, but the trend is clear. . . 3 In Medieval terms, results about H(x) are properties of the size of spells, they are about the complexity of magic incantations! The idea that every- thing is software is not as new as it may seem. 3 One current project is to build a 3D printer that can print a copy of itself. See http://reprap.org.
  • Bibliography [1] A. Burks, Essays on Cellular Automata, University of Illinois Press (1970). [2] G. J. Chaitin, Meta Math!, Pantheon (2005). [3] E. Codd, Cellular Automata, Academic Press (1968). [4] A. Coudert, Leibniz and the Kabbalah, Kluwer (1995). [5] F. Dyson, The Sun, the Genome and the Internet, Oxford University Press (1999). [6] F. Dyson, A Many-Colored Glass, University of Virginia Press (2007). [7] U. Eco, The Search for the Perfect Language, Blackwell (1995). [8] E. Fredkin, http://www.digitalphilosophy.org. [9] F. Hoyle, Ossians Ride, Harper (1959). [10] J. von Neumann, Theory of Self-Reproducing Automata, University of Illinois Press (1966). [11] L. Smolin, Three Roads to Quantum Gravity, Basic Books (2001). [12] S. Wolfram, A New Kind of Science, Wolfram Media (2002). [13] K. Zuse, Rechnender Raum (Calculating Space), Vieweg (1969). 43
  • 44 Chaitin: Metabiology
  • Chapter 4 The information economy S. Zambelli, Computable, Constructive and Behavioural Economic Dynamics, Routledge, 2010, pp. 7378. In honor of Kumaraswamy Velupillais 60th birthday Abstract: One can imagine a future society in which natural resources are irrelevant and all that counts is information. I shall discuss this possibil- ity, plus the role that algorithmic information theory might then play as a metatheory for the amount of information required to construct something. Introduction I am not an economist; I work on algorithmic information theory (AIT). This essay, in which I present a vision of a possible future information economy, should not be taken too seriously. I am merely playing with ideas and trying to provide some light entertainment of a kind suitable for this festschrift volume, given Velas deep appreciation of the relevance of foundational issues in mathematics for economic theory. In algorithmic information theory, you measure the complexity of some- thing by counting the number of bits in the smallest program for calculating it: program Universal Computer output. If the output of a program could be a physical or a biological system, then this complexity measure would give us a way to measure of the diculty of 45
  • 46 Chaitin: Metabiology explaining how to construct or grow something, in other words, measure either traditional smokestack or newer green technological complexity: software Universal Constructor physical system, DNA Development biological system. And it is possible to conceive of a future scenario in which technology is not natural-resource limited, because energy and raw materials are freely available, but is only know-how limited. In this essay, I will outline four dierent versions of this dream, in order to explain why I take it seriously: 1. Magic, in which knowing someones secret name gives you power over them, 2. Astrophysicist Fred Hoyles vision of a future society in his science- ction novel Ossians Ride, 3. Mathematician John von Neumanns cellular automata world with its self-reproducing automata and a universal constructor, 4. Physicist Freeman Dysons vision of a future green technology in which you can, for example, grow houses from seeds. As these four examples show, if an idea is important, its reinvented, it keeps being rediscovered. In fact, I think this is an idea whose time has come. Secret/True Names and the Esoteric Tradition In the beginning was the Word, and the Word was with God, and the Word was God. John 1:1 Information knowing someones secret/true name is very important in the esoteric tradition [1, 2]: Recall the German fairy tale in which the punch line is Rumpelstiltskin is my name! (the Brothers Grimm). You have power over someone if you know their secret name. You can summon a demon if you know its secret name.
  • The information economy 47 In the Garden of Eden, Adam acquired power over the animals by naming them. Gods name is never mentioned by Orthodox Jews. The golem in Prague was animated by a piece of paper with Gods secret name on it. Presumably God can summon a person or thing into existence by calling its true name. Leibniz was interested in the original sacred Adamic language of cre- ation, the perfect language in which the essence/true nature of each substance or being is directly expressed, as a way of obtaining ultimate knowledge. His project for a characteristica universalis evolved from this, and the calculus evolved from that. Christian Huygens, who had taught Leibniz mathematics in Paris, hated the calculus [3], because it eliminated mathematical creativity and arrived at answers mechani- cally and inelegantly. Fred Hoyles Ossians Ride The main features in the future economy that Hoyle imagines are: Cheap and unlimited hydrogen to helium fusion power, Therefore raw materials readily available from sea-water, soil and air (for example, using extremely large-scale and energy intensive mass spectrometer-like devices [Gordon Lasher, private communication]). And with essentially free energy and raw materials, all that counts is technological know-how, which is just information. Perhaps its best to let Hoyle explain this in his own words [4]: [T]he older established industries of Europe and America. . . grew up around specialized mineral depositscoal, oil, metallic ores. Without these deposits the older style of industrialization was completely impossible. On the political and economic fronts,
  • 48 Chaitin: Metabiology the world became divided into haves and have-nots, depend- ing whereabouts on the earths surface these specialized deposits happened to be situated. . . In the second phase of industrialism. . . no specialized deposits are needed at all. The key to this second phase lies in the pos- session of an eectively unlimited source of energy. Everything here depends on the thermonuclear reactor. . . With a thermonu- clear reactor, a single ton of ordinary water can be made to yield as much energy as several hundred tons of coaland there is no shortage of water in the sea. Indeed, the use of coal and oil as a prime mover in industry becomes utterly inecient and archaic. With unlimited energy the need for high-gra
Popular Tags:

Click here to load reader

Reader Image
Embed Size (px)