Absurdity and Meaning in Cognitive Science

 by Richard Hodges

Copyright © 1998
 

Frosty the Snowman
Is a fairytale they say
He was made of snow
But the children know
How he came to life one day

-Christmas song
 

Could a computer ever be conscious? There are noisy arguments on both sides of this question, arguments that usually seem absurd, driven by preconceived ideological commitments rather than by critical thinking.

The common-sense position is that people are in fact conscious and that their awareness of their own consciousness and their belief in it is valid, but that  no existing computer or program  is conscious. Assuming a common-sense definition of “consciousness,” this is undoubtedly a true statement.

But there is a school of thought that would go beyond common sense to add that it is absurd to attribute consciousness to a computer no matter how sophisticated its construction or behavior.  Consciousness cannot arise from an unconscious material object, this argument maintains. There is a group of writers who work various versions of this argument. Searle’s “Chinese Room” is an influential example of the genre. Searle is a very important contemporary philosopher of mind, with wide-ranging and deep if somewhat tendentious critical analysis of many of the loose ideas about mind that are bandied about these days (see for example his book The Rediscovery of the Mind: Representation and Mind), but he is most famous for this “Chinese Room” thought experiment. Searle asks us to suppose we have a computer program that can translate Chinese as well as a human translator. A claim could be made that the computer consciously “understood” a sentence that it was translating. But Searle asks us to imagine that instead of an electronic computer, we had a roomful of non-Chinese speaking people each carrying out different steps of the program. The room could then translate Chinese just like the computer could. But then, is it possible to say that the room “understands” Chinese? Certainly none of the people in the room understands Chinese as a result of carrying out any kind of computer-like instructions, and Searle asserts that it is therefore absurd to say that the room understands Chinese.

This kind of argument depends on an assumption that we have a well-founded definition for “consciousness” and that we have an understanding of it that is basically correct. In fact the only real datum we can start with is our apperception of having a private world of subjective impressions. As inheritors of a certain culture of thinking about ourselves, we tend to extend this to a concept of having an individual consciousness. This is a step of abstraction. Many of the commonly supposed properties of this “consciousness” are not actually given in the data. For example, the most important property that consciousness is supposed to have is that it presents to us a realistic and coherent account of our environment, internal or external. But as has been pointed out by many careful observers, this is simply not true, at least it cannot be counted on. At the very roots of Western philosophy there is Plato’s cave analogy, where he maintains that what we experience consciously, as it were, is but a deceptive shadow of real things. At the opposite end of intellectual history, Freud showed us how what we experience is in many parts if not in main substance a facade put up by the unconscious mind to keep us from becoming aware of feelings that are supposed to remain suppressed; and not only in dreams—our waking experience is more like a dream than we like to accept. There is, it appears, a problem with common sense.

So how do we know exactly what it really is to “understand” a sentence in Chinese? Perhaps the apperception “I understand” is generated by some internal organ of the mind to put a simple gloss, a graspable handle, on what is much like the results of a complex program of syntactic and semantic manipulations, more complex than any we presently know how to program into a computer, and involving elements such as sensation memories that we do not now know how to program at all, but differing little in fundamental nature. Searle’s syllogism starts with the premises 1) some people can “understand” Chinese, and 2) a “Chinese Room,” or a computer, cannot, because it is made of components that do not understand Chinese. But the second premise is an unjustified leap because it does not acknowledge the fact that we do not understand what it means to “understand”; we only experience it. This of course does not yet prove that Searle’s conclusion is necessarily wrong, only that the means by which he arrived at it cannot be trusted.

A number of other arguments against computer consciousness have been made. A highly informed trip through most of them (including Searle’s argument) is given in Robert Penrose’s best-selling books The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics (Oxford U. Press, 1989) and Shadows of the Mind : A Search for the Missing Science of Consciousness (Oxford U. Press 1994). These arguments are summarized more compactly and rebutted by other contributors including Stephen Hawking in a volume based on the 1995 Tanner Lectures The Large, the Small, and the Human Mind (Cambridge U. Press, 1997). These books are surprisingly popular considering that they are filled with intricate argument based on highly technical and arcane material from several fields of science. Penrose is a brilliant expositor as well as a genuine contributor to several fields of science and mathematics. But there are problems similar to the problems with Searle: in pursuing his agenda, which is to promote a certain concept of consciousness, Penrose puts a spin on every argument that ignores crucial ambiguities and pretends that we have a clearer idea what we are talking about than we do. The very brilliance of his presentation obscures this fact, and should alert us to be even more careful than usual.

Let us examine three of Penrose’s basic beliefs. First, he adopts a version of Platonic idealism with respect to physical reality: mathematics, he claims, is the fundamental reality underlying the physics of space, time, and matter both on the large scale (the structure of the universe, described by Newtonian physics, thermodynamics, and general relativity) and the small scale (the nature of fundamental particles and forces, described by quantum mechanics). Much of the known mathematics that describes these phenomena has been proven by many experiments to be astonishingly accurate, and Penrose accepts this as evidence that this mathematics is in fact the true underlying reality. Moreover, calling himself a “physicalist,” he believes that physics and the essentially simple mathematics of which physics is the phenomenal expression exhaust the principles necessary to explain all manifestations of matter, including life and consciousness.

Penrose’s second main idea addresses the well-known fundamental incompatibility between relativity theory and quantum theory that prevents them from being unified. Numerous heroic efforts to develop a mathematical theory that unifies these two have been advanced, but all have serious flaws: either they have been proven wrong by experimental discoveries, or the mathematics at some point becomes too difficult so that the proposed theory is really just a sketch. One of the main difficulties is understanding what quantum theory really means—its terms are really very alien to some of our ordinary intuitions about the world.  The famous quip of Neils Bohr, one of the founders of quantum theory, is still apt, that “it seems to work whether you believe in it or not,” was a deliberate comparison between quantum theory and superstitious magic.

An idea in quantum theory that is particularly difficult to understand is “superposition”—according to the standard mathematical formulation of quantum theory, an object can in some sense be in a state that is indeterminate between two or more discrete definite states, yet it always interacts as if it were in one of its possible discrete states. The process of selection of which among the possible states is the “real” one is  the so-called “observer effect”: the act of examining a quantum superposition state, or of interacting with such a state, appears to have a mysterious effect that in some way changes the nature of reality from subsisting in some sense simultaneously in several different states, to being in only one of these states. As Penrose points out, no really satisfactory solution to this problem is known; many different proposals have been made and argued with great subtlety, but each proposal has serious problems. He reviews this spectrum of interpretation and provisionally adopts one particular position, that quantum state reduction is an objective physical phenomenon of unknown nature, not just an illusion depending on point of view, nor just a manner of speaking, which are the essential positions of other proposals.

He then proposes as a mechanism his own idea that the reduction of superposition states happens because of the interaction of gravity. In other words, the curvature of space-time that general relativity tells us is what gravity consists of imposes a sort of tension that causes a quantum superposition of states to become unstable. As Penrose observes, to make this idea comprehensible requires new physics that we do not yet know, physics precisely in the area where relativity and quantum theory have a discrepancy. This is an imaginative idea, but most physicists would say that it is too premature to make such a specific proposal, and anyway there are solid reasons for doubting it. Among the difficulties of such a proposal is that it would have to act in an incredibly tiny scale of time and space, the “Planck scale” given by certain ratios between the fundamental constants of gravity, light, and quantum theory. In the space dimension, this scale is nineteen powers of ten smaller than the smallest subatomic particles such as protons, and it is very difficult to imagine any possible experiment that would reveal processes on this scale, or to see how such processes could generate macroscopic effects such as Penrose’s supposed “objective observer effect.” Perhaps the greatest value of Penrose’s proposal is that it has spurred some interesting but so far unsuccessful efforts in theoretical physics.

It is here that we find a first reason to be suspicious about Penrose. Is it his faith in an essentially simple underlying mathematical reality that makes him want to look for the missing key in this corner of physics where general relativity and quantum theory meet, a corner that is apparently well-lit by the important new ideas in the theory of “black holes” developed by Penrose himself and his close colleagues such as Hawking? He seems to be hoping that these two by now almost classical theories will explain each other’s conundrums, rather than searching in the darker and more maze-like corridors of superstrings and bootstrap dynamics and theories yet unknown.

But Penrose does not content himself with speculation in his real fields of physics and mathematics, he uses this as a springboard to address the question of consciousness. Here, Penrose is alleging that consciousness is is based on quantum phenomena. This proposal can be placed within a contemporary trend of thought in which a number of less sophisticated  thinkers have tried to establish a quantum explanation for consciousness. Most of these proposals seem motivated by certain not-very-well understood properties of quantum systems that seem to entail a connection between physical objects that are at a distance from each other, allowing a possible explanation for extra-sensory phenomena, and possibly offering an explanation for “holistic” properties of consciousness. It would be a fair summary of the response by the great majority of researchers in cognitive science, in neuroscience, and in physics is that currently, in spite of a great deal of effort, none of these proposals have been worked into a theory well-formulated enough to really explain anything, and that no experiments have revealed any effects that justify such proposals, largely because no testable experimental predictions have been made. Further, most physicists agree that it has been proven both theoretically and experimentally that the putative quantum connection that exists between separated objects is not a “real” connection but is only an artifact of the mathematical formalism. This is well-understood by Penrose, but seems to have been missed or deliberately obfuscated by new-age writers such as Capra and Bohm.

What seems to give Penrose a little more substance than other such quantum consciousness ideas is that he alleges that specific new physics underlies this connection. To him, the old quantum physics, as known today,  is not adequate: in it the only “effect” that can be demonstrated is that two observers who cannot otherwise communicate with each other will have a correlation between what they observe in a certain experimental setup that cannot be explained “classically,” without quantum theory. But it has been proven rigorously that this correlation cannot be used for communication—it is no more surprising really than that two people on opposite sides of a city see a flash of lightning at the same moment.  However, if the observer effect is objective, as Penrose believes, it is not merely a notational artifact and would give rise to presently unknown effects, though as mentioned earlier these effects may be on too small a scale for us ever to observe them. But here we find a double problem in going along with Penrose—questionable new physics as the explanation for a questionable approach to consciousness.

But this does not exhaust the difficulties with Penrose. His real project in these three books is a new theory of mind and consciousness. His fundamental problem appears to be an ideological dilemma—he describes himself as a “physicalist”, in other words he believes that all phenomena including consciousness take place as a result of the operation of mathematically lawful physical processes, yet he is opposed to “computational” views of mind. He defines his position this way: “Appropriate physical action of the brain evokes awareness, but this physical action cannot even be properly simulated computationally.”  He is at a sharp dilemma here because one would ordinarily believe that any physical process that can be described accurately by mathematics can be simulated by computing the mathematical equations. So he has several difficult things to accomplish: to attack the idea that mind can be realized computationally; to imagine a theory of mind that would be physical; to show that the physics involved in this model cannot be simulated computationally.

On the issue of a new physical theory of mind Penrose’s achievements are not serious. He does propose some neurophysiological models that purportedly open a door for non-local quantum-mechanical action. While some of these are interesting (for example his proposal that the molecular microtubules that are observed in large numbers in neurons may organize a protected molecular environment in which long-range quantum processes could operate, analogously to superconductivity and other similar well-known effects) they are highly speculative and have not been confirmed in the laboratory, as he is aware.

On the issue of attempting to show that his new physics of mind cannot be simulated computationally, Penrose also makes some gestures, but here he always seems to confuse computability of consequence from cause given some known law of cause and effect, with being able to computationally recognize whether a proposed consequences may follow from a known law given randomness of causes, which is a much more difficult problem. This issue is central to the mathematical theory of computability and Penrose understands it correctly in this context (see the section below for a discussion of how Penrose applies this theory), but he does not seem to be able to apply it consistently to some of his thought-experiment universes, mathematical constructs in which the second problem is known not to be solvable, whereas Penrose writes sometimes as if it were the first problem, and as if these thought-experiments proved that in the real universe there are certain laws that one cannot compute in the first sense.

His attack on computationalism of mind is more serious. One of his arguments is of the Chinese Room type, discussed above, and Penrose contributes little new to this line. His other main attack is based the mathematical theory of computability. In this theory one of the most striking results, which has bemused many thinkers with its seeming paradoxicality,  is Godel’s theorem which says essentially that no closed computable mathematical system is complete: there will always be true mathematical facts that can be expressed in the terms of the system but cannot be proven within it. From this, Penrose leaps to the conclusion that since the human mind can prove some of these facts, it must have some properties that lie outside of any such system. There are several problems with this conclusion though, most notably the premise. “To prove mathematical results within a closed system” is not a particularly good model for consciousness. For one thing, even the mind of a mathematician does not work, either consciously or unconsciously, by formalistic methods of proof such as are entertained in Godel-type arguments.  And even when a mathematician does use formalistic systems, perhaps as a post-creative process for validating and documenting his intuition, he is not confined “within” them, rather he is their creator, he crafts them as needed for his purposes. The creative process of the mathematician is similar to that of other creative activities, and involves maintaining awareness of a ramifying network of interdependent difficulties, organizing motivation and effort in relation to these problems, using mental imagery to create representations of these problems and solutions to them, and communicating these solutions through language and and other productions. This is no different in essence than any of the sort of social processes than the human frontal lobes evolved to deal with.

There is another important objection to the Godel-type arguments against computationality of mind. This is that the resources, the various levels of the data storage hierarchy and the number of computations per second, of real computers are finite and actually quite limited. This is different from the abstract computers imagined in the mathematical theory of computability which have infinite memory and which have an arbitrarily long time available to complete their calculations. One common way of defining such an infinite abstract computer is called the Turing machine, which Penrose invokes frequently.  The point is that the calculations that are possible for real machines are very much more limited than for Turing machines.

One of the examples given by Penrose illustrates a similar point. Since chess is a game with a finite number of rules, it is easy to program a Turing machine to play a perfect game, by having it go through all the possible moves and counter moves and so on, and determine exactly which move is the best. But to do so would exhaust a real computer as large as the whole universe. People do not use this method to play chess or to do mathematics. The current computer chess champion, “Deep Blue” does use such brute force methods, and while it is not infinite, it can examine many millions of possible chess moves per second. But it can be beaten by a human, who examines far fewer possible moves and uses quite other methods. Any realistic computational model of mind would have to also. Much work has been done to develop programs that play chess or do mathematics, using methods more human-like than the exhaustive methods envisioned by Godel or Penrose, and while these programs are not nearly as good as a human they do have some properties that seem similar to “insight,”  “analogy,” “understanding,” presumably crucial human mental functions. These are among the properties of human mind that Penrose insists require explanations from unknown physics rather than from unknown programming methods.

In short, Penrose seems to have an ideological agenda to prove the existence of some kind of consciousness that is inexplicable by present computational, neurologically based models. Such models are in sober fact at their present stage of development very far from getting at the essence of consciousness or mind. But Penrose does not address them on their own ground, where they could be attacked successfully—his most serious forays are from fields of pure mathematics that are far removed, and in them he shows the naivete of an outsider. Even when he tries to turn to mathematical physics he seems mainly to be trying to trade on his own substantial reputation to buttress arguments of even weaker relevance.
 

The Search for Meaning in Cognitive Science

Now let us look at the other side of the question of computer consciousness. On this side are the research communities of artificial intelligence (AI) and its sister field cognitive science. Cognitive science attempts to study the human mind by experiment and analysis, to understand what the mind is and how it works from a point of view essentially like physics and chemistry. Artificial intelligence attempts to create computers that do things that would seem to require intelligence. The history of these fields, which began alongside the beginnings of computers in the 1940’s,  shows a series of generation-long boom-and-bust cycles. The boom phase always begins with a chorus of naive projections that it  is only five or ten years away to be able to make a computer do some kind of task that requires human intelligence, and that achieving this would have major benefits for humanity. Funding and research activity heat up. In a few years, preliminary achievements are claimed but demonstrations fail to convince. Disillusionment sets in and the field retreats to a more contemplative mode. The chorus chants that what we need now is fundamental research in cognitive science, to understand better how human minds work. Finally, at length, painful memories fade, past failures are explained, new findings in science and new developments in technology give fresh hope, a new generation of money becomes hungry for the big win, and the cycle begins again.

Careless claims aside, no respectable scientist believes that we now have a genuine mechanistic theory of mind, or that we know how to make a computer that “thinks” or is “conscious” in a way that can be considered human. Many in the field are thoughtfully aware that we do not really understand deeply enough even what it really is to “think” or “be conscious.” Nevertheless, most in the field, and many other people besides, believe as an article of faith that implementing both these properties in a computer is inevitable and desirable. Several factors are at work in reinforcing these beliefs: a desire to free people from certain labors by technology; the hubris of science as a universal means of knowledge; a commitment to the inevitable destruction of myths inherited from pre-scientific dogma (here the myth of a non-material soul supposed to underly the experience of self); the joy of creating and playing with complex new toys; the wish to pioneer into unexplored areas of knowledge of importance for man’s understanding of himself; and the tendency to see a universal metaphor in whatever new impressions one is exposed to in one’s life and work, in this case the computer. Each of these factors carries a baggage of controversy, and the combination of them focuses a particular cultural energy on the question of the computer and the mind.

The last factor mentioned above, the power of experience to form new metaphor, is very important. Many historical ideas about what nature is like, including human nature, show this kind of origin. There are certain well-known examples: the Newtonian theory of movements of the planets made the solar system seem more real and led to a renewal and revisioning of of ideas about planetary influences and “the music of the spheres”. The quantum theory of the atom, an epochal discovery one of whose results in technology is the semiconductor revolution, has found a contemporary echo in “the physics of consciousness,” as in Penrose’s work discussed above. Earlier, the industrial revolution led to a phase of mechanistic thinking about man, in which man was compared to a mechanical device, or to a factory, or to a steam plant in which pressures build up in certain places and have to be released (this metaphor is expressed clearly in Freud for example). One could carry this kind of analysis further; probably many of the ideas inherited from tradition about man’s place in the order of things can be traced to metaphors derived from different historical facts that at one time or another dominated the horizon of daily life: sedentary agrarian and nomadic pastoral ways of life; the experience of conquest and defense; the hierarchical state organization; cities and their interpenetrating interdependent specialist communities.

Like these examples, the contemporary experience of working with the computer is a powerful and consuming one, and without doubt informs some of these new ways of looking at man. One common experience for those who work with computers every day, especially for craftsmen programmers, is the experience of the computer as a sort of quasi-human with whom one has a relationship. This happens most often at the end of a long process of getting a new program to work, a process that has been compared to a war in which one must fight and win many battles against errors in one’s own code, misunderstandings of how things work, and bugs in the system itself. After a period of struggles that may seem Sysiphian, one is finally able to stand back and admire the finished program ticking away, flawlessly meshing with the hidden machinations of the system of which it is now a tiny but salient part. This can be an almost mystical moment. Even though we “know” intellectually that the computers and the networks we work with every day do not have minds or feelings, in some strange way we may feel that we have come to have an emotional relationship as if to another being.

And what is mind anyway, what is being? Are these such an objective things that we can be sure that only humans have them, and perhaps animals? What about God, and other spiritual entities, do they exist as beings with some sort of sentience? We may or may not assent intellectually to believing in these, and we may or may not have direct experiences where we feel their presence and beingness; but surely many if not most people do have such experiences, and these experiences are crucial to the meaning of the religious and metaphysical ideas we have inherited. If the game of the forest and the crops of the land can be known as sacred beings with whom our ancestors communed through mystical rites, if the stars and planets that seem to regulate the seasons can be worshiped as gods, if there was once a holy state with a divine king, if post-Newtonian man can think of mechanical nature as sacred and science as her religion, if post-Darwinian man can feel the biosphere and the evolving tree of living things as a new kind of god, if money can become the god of what is effectively a new religion called capitalism, which has a sect called communism that worships the God industrial labor, why isn’t it legitimate that the computer be felt as some sort of half-sentient being, the net as some as possessed of some sort of half-divine spirit? Legitimate or not, that is the way we find ourselves feeling, and these feelings partly underly the faith and the hope that the mind in the machine will be fully realized one day.

Is mind, consciousness, anything at all other than a perhaps a mere metaphor? If we can question the consciousness of a computer or of a planet, if we can question the existence of various gods or of God, shouldn’t we entertain the possibility of questioning our possibly naive belief in the existence of our own mind? There is an honorable tradition of such questioning. For example, in certain important schools of Buddhism, there is the idea that it is necessary to investigate and by careful analysis to eradicate the belief in the existence of oneself as a separate, conscious, being. Many traditions teach the necessity of death of the “ego,” emphasizing to various degrees the intellectual error of false belief in something imaginary, or the emotional sickness of attachment to self-interest, self-concept, and so on. A tradition elaborated in The Gurdjieff teaching explains this with the idea that man’s being operates as a swarm of mini-individualities each of which in turn has its moment upon the throne of self and then feels “I am” and takes itself wrongly to be the whole.

The process of all such questioning makes it clear that the question of consciousness itself is not well-understood. We do not know at a fundamental level what it would mean to be conscious, or not to be. We do not have real categories of thought to discriminate  in a useful way the different levels and kinds of processes for which words like “mind” and “consciousness” are obliged to do stand-in service.  The most important thing that the question of computer consciousness may be doing in the present world is to make it more urgent that there are questions of this sort that need to be thought about in a new way.