1. A Date
In December 1965, the RAND Corporation printed a paper numbered P-3244. It was called “Alchemy and Artificial Intelligence.” Its author, Hubert Dreyfus, was a philosopher. On pages three and four he copied out, item by item, something from eight years earlier.
Those were the four predictions Simon read aloud at a dinner at a Pittsburgh hotel in November 1957. Essay seven starts that night. Within ten years a computer would be world chess champion; within ten years a computer would discover and prove an important new mathematical theorem; within ten years a computer would write music critics would accept; within ten years most theories in psychology would be written as programs.
After listing all four, Dreyfus wrote that it was still too early to put the claims to a conclusive test. Then he gave a date. In November 1967, at the ten-year mark, he said, they could put the original expectations beside what had actually happened. It was an appointment to check the books.
At the end of essay seven, we hadn't found a written reckoning in which the two authors went through their predictions one by one. But two years before they came due, someone outside the field had set down the due date in black and white. This essay is about that person and what became of the reckoning he proposed.
There is a footnote on the first page of P-3244. The paper is based on an informal talk given at RAND in August 1964, it says, and the views are the author's own. RAND printed such papers as a convenience to its staff. This one was a talk made into a paper and circulated inside RAND.
A later account says RAND suppressed it. That account comes from a book written more than a decade later. In Machines Who Think, published in 1979, Pamela McCorduck records the recollection of Paul Armer at RAND. He had had reservations about publishing it. His colleague Robert Reinstedt pushed it through on the grounds that RAND shouldn't refuse to publish a piece of its own research just because it disliked the conclusions. In December 1965 it went out in print as P-3244. There was hesitation, and then it was published. What traveled was that it was suppressed.
Reinstedt is named in the acknowledgments to the paper itself, along with Stuart Dreyfus. Stuart was his brother. More than twenty years later the brothers would put their names to an article together.
2. Four Kinds of Activity
The way P-3244 poses its question is where many of the later stories start to go their separate ways. Dreyfus didn't ask whether a machine could think. In the introduction he narrows the question considerably: can digital computers be programmed to exhibit the simple intelligent behavior of a child or an animal? Play chess, solve simple problems, read sentences, recognize patterns. Those are the four areas Part I reviews: chess, problem solving, language translation, and pattern recognition. The heading of Part I is “The Current State of the Field of Artificial Intelligence.”
Part II deals with what those difficulties mean and takes up human and machine information processing. Here he introduces three terms for ways people handle information differently from digital computers. One is fringe consciousness: while someone is paying attention to one thing, what lies around it but isn't being looked at directly is also at work. Another is distinguishing the essential from the accidental: people can tell what matters in a situation from what just happens to be there. A third is tolerating ambiguity: people can carry several possible meanings of a sentence forward and let the context decide.
The summary also identifies a premise on the other side. The optimists of the time, he writes, believe intelligent activities differ only in complexity. That belief, together with some early successes, convinced them that the information processing behind any cognitive performance could be written as a program and simulated on a digital computer. That was the premise he wanted to question.
He divides intelligent activity into four kinds. The summary says two of them don't require those three human ways of handling things, so they can be programmed by the methods of the time. Major progress on the other two, it says, will have to wait for a radically different kind of computer. The only prototype then available is the human brain, which we do not yet understand.
Part III divides the obstacles to programming into three. The headings name the infinity of facts, the indeterminacy of needs, and the reciprocity of context. The infinity of facts means that if you list the facts you would need to get something right, the list doesn't end. The indeterminacy of needs means that what counts as relevant depends on what you're trying to do, yet to work out what you're trying to do you first have to know what is relevant. The reciprocity of context means that context decides what a word means, while the words decide what the context means. In one case the list never ends, in one the question leads back to another question, and in one you go around in a circle.
What these ninety-odd pages have given us so far is an argument about particular programs, particular predictions, and particular problems. There are two points to check later against what happened. One is the date: November 1967. The other is the claim in the summary that two kinds of activity will have to wait for a radically different computer. Each has a different sequel.
3. A Ten-Year-Old
After P-3244 came out, people in the field answered. In 1968, Seymour Papert wrote a seventy-six-page laboratory memo. His name has appeared in both preceding essays: the book on perceptrons was by Papert and Minsky. The memo was called “The Artificial Intelligence of Hubert L. Dreyfus: A Budget of Fallacies.” The title was the answer.
The two documents are mirrors in form. Dreyfus's is a RAND paper printed by an institution for its staff. Papert's is an internal memo from MIT's Artificial Intelligence Laboratory, also printed by an institution for its own people. An internal document from one answers an internal document from the other. Neither is a journal article. Neither went through outside review. From the outset the dispute was fought on two institutions' mimeograph machines.
Simon's reaction was a sentence: he called the thing printed in RAND's name garbage.
The reaction on the outside was different. On June 11, 1966, The New Yorker ran a short piece on Dreyfus for general readers. At the time, it said, no computer could yet play a respectable game of chess. Not yet. Not never.
Then there was a headline. The story later appeared in the newsletter of the Association for Computing Machinery's AI special-interest group. The headline had two parts. The first said a ten-year-old could beat the machine and attached Dreyfus's name to the remark. The editor added the second: but the machine can beat Dreyfus. The headline was about one game.
4. The Game
The opponent was a chess program called Mac Hack VI. It was written by Richard Greenblatt at MIT. He had taken a chess program the school already had and added some fifty rules of thumb. It ran on a DEC computer, the same kind of machine at the same school as the blocks program in essay nine. In 1967 the program entered tournaments against people. It had a rating in the neighborhood of 1400.
Dreyfus played white, the machine black. They each opened by moving the pawn in front of the king ahead two squares. The machine won. Papert had arranged the game.
Simon later said it was a splendid game between two players who weren't very good, in doubt to the end. The machine got out of a bad spot with a series of checks and a queen fork, then won. Two players who weren't very good. The man who said it had written the four predictions with Newell.
Put the game beside the date again. In 1957 the first prediction was that a computer would be world champion within ten years. In 1965 Dreyfus proposed checking it in November 1967. Around the time it came due, the machine sitting across the chessboard had a rating of about 1400. That month the world champion was still a person. The first prediction had not come true. But that is not the entry everybody remembers. What they remember is the machine beating Dreyfus.
The question Dreyfus asked was whether the machine was world champion by the deadline. The game answered whether a machine could beat Dreyfus. The second was a much easier question, and he hadn't asked it.
Essay five described a way to replace a question that couldn't be measured with a game that could. The person who posed the question made the change himself and said in print that he was doing it. This time a reckoning was replaced with a game of chess. The person who made the change wasn't the person who asked the question.
Dreyfus's own protest survives. He wrote to the newsletter, not to say the machine cheated or the game didn't count, but to say it was being used to refute something he hadn't said. In his view, people had turned a criticism of the programs of the time and a challenge to a ten-year deadline into the claim that a computer could never play chess. Then they used a game to refute that claim. Simon answered the protest: Dreyfus had written with a pen that had teeth, so he ought to expect the people it bit to bite back. By then neither of them was talking about chess.
5. Four Assumptions
In January 1972 he published a book, What Computers Can't Do: A Critique of Artificial Reason. He rearranged the material from the 1965 paper. Part II of the book has four chapters, one assumption to each. They are the biological, psychological, epistemological, and ontological assumptions. What does each say?
The biological assumption: at some level, usually taken to be that of the neuron, the brain processes information in discrete operations, through a biological mechanism comparable to a switch in a digital machine.
The psychological assumption: the mind can be viewed as an information-processing device that follows formal rules. In its stronger version, these are the rules that actually produce behavior.
The epistemological assumption: all knowledge can be formalized. He separates this into two claims. Behavior can be formalized, and the resulting formal system can be used to reproduce the behavior.
The ontological assumption: in principle, all the information about the world that matters to intelligent behavior can be broken into elements independent of the particular situation. The world is then taken to be a collection of logically independent facts.
With all four on the page, what does he ask for? Here is a qualification that almost completely disappeared. He doesn't say he has proved the four assumptions false. He says people in the field often treat them like axioms that guarantee the result. He wants to put them back in the category of hypotheses, where the success or failure of the research can test them. From axioms to hypotheses. Not from axioms to fallacies.
There is a similar distinction in the chapter on biology. He separates the claim that the brain resembles a digital computer on some level from the claim that the field's heuristic programs will succeed. Even if the first holds, the second doesn't follow. In the chapter on knowledge he distinguishes formalizing a capacity from giving a causal account of how a person actually exercises it. He isn't attacking every use of mathematics to describe things. He's attacking the leap from a formal description to how an intelligent process actually works.
Chess has a place in the book, too. He groups chess and Go with complex formal systems. In principle they can be formalized, but the number of combinations swells as the elements multiply. In practice exhaustive search is out of the question, so one has to use heuristics. The 1965 paper makes the claim a little stronger. In some positions a human can set aside what doesn't matter and win, whereas the heuristic programs then envisaged couldn't. He put chess and Go in a class susceptible to only limited programming. That isn't the same as saying a computer would never play chess well. In 1972 he still put chess among things that could in principle be formalized.
There are strong judgments in the book. Having examined the evidence then available, he says further major progress in cognitive simulation or artificial intelligence is extremely unlikely. He puts the burden of proof on those who are still optimistic. This is a strong claim. Then he adds a qualification. How much farther faster machines, better programming languages, and cleverer heuristics can push the boundary is something only experiment can decide. There is no demonstrable technical ceiling here. There is a judgment and a statement that hands it over to experiment.
There is a passage in the conclusion almost nobody quotes. Even if a digital intelligence fully formed from birth, like Athena, were in principle impossible, he says, it would still be necessary to ask how closely the field's techniques, piece by piece, could approximate intelligent human behavior. A fully formed artificial person and a series of local successes aren't the same proposition to him.
6. His Own Accounts
The book came out twice more, and each time he added a preface. The 1979 revised edition had a new subtitle, The Limits of Artificial Intelligence. The long new preface explains how he revised it. The main text stayed, except for individual passages that had come to seem unclear or misleading. The new material and new conclusions from the ten years between 1967 and 1977 went in the long preface. The old book was largely left alone. The new accounts went in front.
There is a new claim in the long preface. Researchers in the field, he says, have now run straight into the problem of commonsense knowledge. In his view, the broad background on which people rely is chiefly embodied skill and know-how, not just a bigger store of facts.
In 1992 the third edition appeared under the title What Computers Still Can't Do. The title acquired the word still. This edition added another preface before the 1979 preface and the original text, both of which stayed. So the reader in 1992 held a book in three layers. At the center was the text of 1972. Around it was the preface of 1979; outside that was the preface of 1992. One person at three different times, bound in the same book, with the last writing first.
In the 1992 preface he makes a correction of his own. He admits that what he used to say about context as a whole had been vague. His brother Stuart's model of skill acquisition and work on neural networks now gave him the sense that he had a more concrete alternative. In the same preface, immediately after announcing the failure of the approach based on rules and facts, he adds a sentence: its failure doesn't mean artificial intelligence of this kind is impossible. He further qualifies his judgment: at least for the time being.
The other point in P-3244 has a sequel too. For two kinds of activity to advance significantly, he had said, a different kind of computer would have to come along, the brain being the only prototype. In 1997 a machine called Deep Blue defeated the world champion, Garry Kasparov, in a match. It was still a digital computer. That match changed an entry in the book. It didn't settle the entry about everyday context. The dinner in 1957 had set a limit of ten years. Forty years passed between that dinner and the match. Both dates go in the same accounts.
7. A Fork in the Road
In the winter of 1988 the brothers published an article in a journal of the American Academy of Arts and Sciences, “Making a Mind versus Modeling the Brain: Artificial Intelligence Back at a Branchpoint.” The opening places two camps side by side. One sees the computer as a system that manipulates mental symbols; the other sees it as a medium for modeling the brain. Then they set out the differences in pairs: formal representation, problem solving, and logic on one side; simulated neural interaction, learning, and statistics on the other.
This is the other end of the line in essay eight. The 1969 book measured the boundary of a kind of machine; the line of work was later said to have died for ten years. In 1988 the two authors put it back on one side of a fork in the road.
They divide the history of the symbolic approach into three periods. From about 1955 to 1965, representation and search. From about 1965 to 1975, microworlds. From 1975 on, the problem of commonsense knowledge. One example in the microworld period is the blocks program from essay nine. The hope at the time, they write, was to make one restricted little world after another more realistic, then fit them together into an understanding of the real world. What happened after 1975, in their account, was that the problem made the researchers face the large expanse of everyday background they had first done their best to get around.
Those periods are the route this series has taken, too. The summer and the decade in essays six and seven were representation and search. The table of blocks in essay nine was a microworld. The third period, common sense, is somewhere this series hasn't reached yet. Someone who criticized the field drew a map of its periods. He drew it from outside and put himself at the entrance to the third. That expanse of everyday background was what he had been pointing to since 1965.
The article is often said to show that Dreyfus predicted neural networks, or even deep learning. That isn't what the end says. The brothers do not announce that neural networks are the answer. They write that neural-network modeling may at last have been given a chance to fail. Then they ask how a network could generalize in a way appropriate to the particular situation.
Farther on they make a list. If a network is to develop from its own experience the kind of associations people make, rather than ones decided for it by whoever trains it, it may have to share some things with people: a sense of what is appropriate, needs, desires, emotions, a body like ours, the ability to act, and the capacity to be hurt. The capacity to be hurt comes last.
The sentence doesn't say the machine will never manage it. It says that to do it the way a person does, it may take more than data or better algorithms. It may take having needs, having desires, and having a body that can hurt. This paragraph too is rarely quoted. The line that gets quoted is the one about the fork in the road.
He was still writing in 2007. In a journal of artificial intelligence he published “Why Heideggerian AI Failed and How Fixing It Would Require Making It More Heideggerian.” There is a word in that title: fixing.
8. Checking the Books
Set down what others entered in his accounts, in order by year. In 1968, Papert: a budget of fallacies.
In 1979, McCorduck records the field's deep hostility toward him. Her own view is not simply a dismissal. She criticizes the mocking style that put off people who might otherwise have been persuaded.
In 1983, Newell. His name has come up several times. One of the authors of the four predictions in 1957, he reviewed the book about perceptrons in 1969. In a 1983 essay on the intellectual history of artificial intelligence, he writes that Dreyfus's objection that context cannot be broken into independent, discrete elements is a genuinely central intellectual objection. The answers given up to then hadn't really reached it. That is one author of the four predictions writing eighteen years after Dreyfus called for a reckoning. He didn't accept Dreyfus's conclusion. He conceded that the question hadn't been answered.
In January 1990, the Churchlands write “Could a Machine Think?” for a popular science magazine. They say that Dreyfus's 1972 book pointed to shortcomings in the demonstration programs for cognitive simulation and to a set of repeated patterns of failure.
In 1993, Daniel Crevier writes a history of artificial intelligence. His wording is limited: time showed some of Dreyfus's criticisms to be accurate and perceptive. Some, not all. He adds that the constructive points might have been picked up earlier if they had been made less aggressively.
In 1996, McCarthy. Reviewing the 1992 edition, he begins by asking how Dreyfus can know that the symbolic approach is a degenerating research program. He says it is hard to see. He faults Dreyfus for not reading recent work in areas such as nonmonotonic reasoning and cites that branch's development as counterevidence that the symbolic approach is losing ground.
Yet in the same review McCarthy puts commonsense knowledge and reasoning at the heart of the field. He lists context, nonmonotonic reasoning, introspection, and levels of description among the work still to do. He says human-level AI is not yet a problem engineering can solve and will probably call for fundamental scientific discoveries. At the end he throws the question back. Can Dreyfus give an example of intellectual behavior precise enough and simple enough to show clearly what a person can do and a computer cannot?
In 2004, the revised edition of McCorduck's book keeps both sets of accounts: the contempt and ridicule of the 1960s, and the later recognition that some of his judgments had value.
Set this sequence beside two things often said later. One is that Dreyfus was proved right about everything. But Newell conceded the question wasn't answered, Crevier said some, and McCarthy was still disputing his conclusions in 1996. The other goes the opposite way: he was proved wrong about everything. Yet McCarthy himself put common sense and context at the field's center and said the human level was still not within engineering's reach. Both phrases leave something out. The first leaves out some. The second leaves out McCarthy's own concession.
The edits in this essay have the old shape, but this time more of them fall on the same person.
He wrote that it was still too early to test the claims, and to look again at the ten-year mark. What traveled was that a computer would never play chess.
He wrote that the thing hadn't been done then. What traveled was that it never would be.
He wrote that axioms should be put back among hypotheses. What traveled was that he had proved artificial intelligence impossible.
He wrote that failure didn't mean impossibility, and confined his judgment to the time being. What traveled was that he said artificial intelligence was bound to fail.
He wrote that neural networks might at last have a chance to fail. What traveled was that he predicted deep learning.
Five edits, not all going the same way. The first four make him sound more absolute than he was. The last makes him sound more prophetic.
In those twenty-seven years he was doing something else. He wrapped the book in layer after layer, leaving most of the old text in the middle, altering a few unclear sentences, and putting a new preface on the outside every dozen years or so. In the prefaces he writes what he said too vaguely, what no longer stands, and what he now puts another way. The book from 1992 is a ledger. Open it and the latest corrections come first. Go farther in and there are earlier corrections. At the center is the man of 1972 being corrected.
This series has used carving and construction to look at the people who make machines. They make a thing, draw its boundary, and still find something left outside. Then they build again. Dreyfus was someone who drew a boundary, and his moved too. Each time it moved, the earlier line stayed in the book.
Back, finally, to the date. In December 1965 he set an appointment on page four of P-3244: in the month of the ten-year anniversary, put the expectations beside reality. The month passed. The first of the four predictions hadn't come true. The machine that could sit at a chessboard was rated about 1400. Yet what people remember is a game he lost and a headline saying a machine could beat him.
A man who called for a reckoning was later made the subject of one. He had wanted to check the four predictions. The entry people remember is the one about him.
The machines are still being built. Each generation says again where the boundary lies.