SHRDLU: Understanding inside a world of blocks

The event and the narrow claim

By August 1970, Terry Winograd, a doctoral student at MIT's Artificial Intelligence Laboratory, had a working program that could hold a typed conversation in English about a small world of toy blocks, carry out commands in that world, and answer questions about what it had done and why. He called it SHRDLU. The dissertation describing it went to MIT's Department of Mathematics on August 24, 1970, supervised by Seymour Papert, as MIT's catalogue record for the 1970 doctoral thesis shows. A revised version dated January 1971 circulated as MIT AI Laboratory Technical Report 235, and the same text was issued as Project MAC report TR-84, a 462-page volume now held by the Computer History Museum. In 1972 it reached a far wider audience as an entire issue of the journal Cognitive Psychology, Winograd's Understanding Natural Language, and in the same year as a book from Academic Press.

Chronicle dates the event to 1971 because the January 1971 report is the form in which the work was read and argued over; the thesis submission came five months earlier, and at least one later history, Nils Nilsson's The Quest for Artificial Intelligence, gives the report's month as February. Nilsson also records the origin of the odd name: the letters were the second column of keys on a Linotype typesetting machine, a nonsense word familiar from MAD magazine, chosen when no good acronym presented itself.

The narrow claim is specific. A program running on a PDP-10, with a vocabulary of about 200 words, could parse a defined subset of English, interpret it against a complete model of a simulated tabletop, plan and display actions, and keep track of the dialogue well enough to resolve pronouns and explain its own goals. It was not a general language understander, and its author never said it was. The broader historical claim, that it reset expectations for what language programs could do and then became the standard illustration of why such programs did not scale, depends on what happened after 1971.

A world small enough to know completely

Why blocks? In a 2023 oral history recorded for the Stanford Historical Society, Winograd recalled that Marvin Minsky suggested children's stories as a domain. Winograd declined, reasoning that such stories lean on a great deal of social and bodily knowledge even when their sentences are simple; his fellow student Eugene Charniak took on that problem instead. Winograd chose the tabletop of blocks that the laboratory's robotics group was trying to manipulate with a real arm and camera. Because vision and arm control were still unsolved, he built what he called a "fake block world" on a display screen, so that his own progress would not wait on the hardware.

The choice matched a research strategy that the laboratory's directors were spelling out at the same moment. In their Artificial Intelligence Progress Report, issued as an MIT memo dated January 1, 1972, Minsky and Papert described "micro-worlds" as deliberately schematic models, a kind of fairyland in which nearly every statement would be literally false of the real world, and said they planned to put a large share of the laboratory's effort into building collections of them and learning how to connect them. They presented Winograd's blocks world as the leading case and noted that his robot existed only as a picture on the computer's screen. The bet was that understanding could be studied in depth inside a domain small enough to be known completely, and that breadth would follow by joining such domains together.

Inside the program: grammar, meaning and deduction as procedures

Technical Report 235 describes a system written in LISP for the PDP-10 under MIT's Incompatible Time-sharing System. Working with a vocabulary of 200 words and a moderately busy scene, it filled roughly 80K words of core memory, a figure that included the LISP interpreter, and in compiled form each sentence took 5 to 20 seconds to process and reply to. Its parts were organized vertically, each able to call the others, rather than as a pipeline. An input routine handled dictionary lookup and word endings, so that "littlest" could be understood from "little". The grammar was written in a special language Winograd called PROGRAMMAR and was based on Michael Halliday's systemic grammar, which treats a sentence as a set of meaningful choices (command or question, yes-no or wh- question) rather than as a tree produced by rewrite rules. Winograd had studied linguistics with Halliday in London on a Fulbright before arriving at MIT, as the oral history records. Semantic programs interpreted each phrase while it was being parsed, a web of semantic categories let the word "in" mean "contained in" for a box but "part of" for a stack, and a display routine animated the arm on a DEC 340 screen in real time. The report also offered the complete program listings, on paper and on DECtape, to anyone wanting to run the system elsewhere.

Deduction ran through Micro-Planner, a pared-down version of Carl Hewitt's PLANNER language that Winograd implemented with fellow students Gerald Sussman and Eugene Charniak. What the program knew about its blocks, such as the need to clear a block before lifting it, was written as PLANNER theorems: small programs invoked by matching the goal they could achieve, which backtracked and tried an alternative when they failed, a mechanism described in Stuart Sutherland's 1973 commentary on AI. The thesis title states the central idea: procedures as a representation for data. A word's meaning, a grammatical rule and a fact about support could all be programs that called one another, so the parser could consult the world model halfway through a sentence and discard a reading that made no physical sense. Nilsson's chapter on natural language processing notes that this procedural style was set against John McCarthy's declarative use of logical formulas and fed a long debate between the two camps, and that SHRDLU's procedures for moving objects resembled the STRIPS operators developed for SRI's Shakey robot.

The dialogue that made the case

What made SHRDLU famous was a sample conversation printed in section 1.3 of the report, with the person's turns numbered up to 47 and indented commentary explaining each capability. It opens with the command "pick up a big red block", which the program can obey only after finding room for a green block that sits on top of it and moving that block away. Told next to grasp the pyramid while three were visible, it replied I DON'T UNDERSTAND WHICH PYRAMID YOU MEAN and did nothing. Asked to find a block taller than the one in its hand and place it in the box, it first stated which object it took "it" to mean, then acted. Two turns later the same phrase, the pyramid, was no longer ambiguous, because the program remembered which pyramid had just been mentioned.

The later exchanges went well beyond parsing. Asked whether the table could pick up blocks, it said no, because its semantic rules marked the question as nonsensical. Asked whether a pyramid could support a pyramid, it said it did not know, since no example was present and it could not examine its own programs to decide. Told which kinds of blocks the user liked, it turned the statements into new theorems and used them to answer later questions. In a chain of "why" questions it climbed its own record of subgoals, from getting rid of a pyramid, to clearing off a cube, to building a stack, until it reached BECAUSE YOU ASKED ME TO. It accepted a definition of "steeple" as a stack holding two green cubes plus a pyramid, then built one when asked. When the phrase "on top of" was ambiguous and its heuristics could not settle it, it offered the user two paraphrases to choose between.

Winograd later credited much of the effect to discourse. In the oral history he said that keeping track of what had already been said, so that a command like "put it down" referred to something, gave the exchange a natural feel that caught people's attention. The contrast with an earlier MIT program was obvious to readers. Joseph Weizenbaum's ELIZA, described in 1966, produced replies by matching keywords in the input to decomposition rules and reassembly rules, with no model of the subject under discussion. SHRDLU's answers were checked against a world it could inspect and change, and its errors were errors about that world rather than about wording.

Contemporary verdicts: an exception, not a turning point

The most quoted contemporary praise came from an unlikely place. Sir James Lighthill's survey of the field for Britain's Science Research Council, published early in 1973, was sharply critical of basic research in robotics and language processing and, according to the archive that hosts the report, provoked a deep loss of confidence in AI among British academics and funders. Yet Lighthill added a postscript on Winograd's "exceptionally good" PhD thesis. He praised its writing, called its table-top conversation already famous, conceded that it had achieved far more than the eye-hand robot projects, and judged its clearest influence to be on psycholinguistics, where it pushed researchers toward knowledge-driven accounts of how people process sentences.

He then drew the opposite conclusion from the enthusiasts. A big machine and highly elaborate programming, he observed, had barely managed a slow conversation about a tiny body of material, and any attempt to stretch the methods to a broad domain of conversation would collide with the combinatorial explosion. One swallow does not make a summer, he wrote, and the thesis left his overall pessimism about integrated robot research unchanged.

Sutherland, defending the field in his published reply, corrected the framing from the other side. The work, he argued, was less novel than Lighthill implied: it gathered ten years of earlier techniques, depended entirely on Hewitt's PLANNER, and used a grammatical approach similar to William Woods's, and its fame owed something to Winograd's unusual skill at explaining his ideas. Both readings hold. SHRDLU was an integration more than an invention, and its persuasive force came from showing the integrated pieces working together in a dialogue that anyone could follow.

The other major language bet of those years shows what SHRDLU traded away. Woods's LUNAR system at Bolt Beranek and Newman, as Nilsson recounts, put an English front end on NASA's database of Apollo moon rock analyses and answered 78 percent of the questions geologists put to it at a conference in January 1971. LUNAR worked on real data for real users, but it could only look things up; it had no world to act on and no model of why a question was being asked. SHRDLU had both, for a world that existed only on its own screen.

Why the blocks world did not grow

Winograd named the limits himself. In the closing section of the 1971 report he listed three directions for future work: its knowledge covered one small, simplified subject, nearly all of that knowledge had been written in by hand instead of being taught to it, and a person had to type rather than speak. He also warned that the steeple episode was somewhat deceptive. A new word could be defined in English only when it could be assembled from concepts the system already had. A word such as "face", for the faces of a block, would require a new concept of a part that is not itself an object, with consequences throughout the world model, and no simple definition could supply it.

That warning points to the structural reason the approach did not scale. Every capability in the dialogue rested on hand-written knowledge tuned to one domain: which objects exist, what supports what, what "on" can mean. Lighthill's objection was computational, about search multiplying as the domain widened. Winograd's later objection, set out in the Stanford oral history, was about meaning itself. In SHRDLU, he explained, a block was simply one of a few kinds of object on the table, but in ordinary English people walk around a block, wedge a block behind a wheel and suffer a mental block, uses tied together by analogy and metaphor rather than by any shared definition. Through the 1970s he and Daniel Bobrow at Xerox PARC built a more flexible representation language, KRL, to capture that open-endedness. By his own assessment it inspired successors such as KL-ONE but never solved the problem, and he came to see extra mechanisms layered on logical representations as no closer to how people actually use language.

The microworld programme as a whole met the same wall. The knitting together of small worlds that Minsky and Papert had announced did not yield a system that understood a large one, because the knowledge that made each world tractable was exactly what failed to transfer.

Winograd's turn away from symbolic AI

The author of AI's most celebrated language program became one of its best-known internal critics, and the oral history traces how. After moving to Stanford in 1973, Winograd joined discussions in Berkeley with the philosophers Hubert Dreyfus and John Searle, critics of AI from two different directions. In 1976 he met Fernando Flores, a former minister in Salvador Allende's government in Chile who had spent three years as a political prisoner until an Amnesty International chapter that included the Stanford professors George Dantzig and Robert Floyd arranged the research post that his release required. Their joint reading turned into a paper, then a monograph.

Understanding Computers and Cognition: A New Foundation for Design, published by Ablex in 1986, drew on Martin Heidegger and the biologist Humberto Maturana to argue that people mostly act through embodied, unreflective coupling with their world rather than by manipulating explicit representations of it, and that programs built from such representations would therefore not reach human-like intelligence. Winograd's own Stanford home page describes the book as a sharp break with the philosophy behind AI research. The field took it seriously enough that the journal Artificial Intelligence ran four reviews of it in February 1987, by Vellino, by Stefik and Bobrow, by Suchman and by Clancey, followed by the authors' reply.

The argument carried a stated hedge. Winograd notes in the oral history that the book did not rule out neural network approaches, which do not depend on articulated representations; it said only that they had not yet worked. His practical conclusion was to stop trying to make computers think like people and to make them work well with people instead. By the late 1980s he described his own field as human-computer interaction, and his home page records that he created and directed Stanford's Human-Computer Interaction Group and its teaching program in the subject.

The afterlife of a pronoun puzzle

One small example from Winograd's work outlived the program. To show that understanding requires knowledge of the world, he discussed a sentence in which city councilmen refuse demonstrators a permit because "they" feared, or alternatively advocated, violence; changing the verb changes who "they" are. As the collection maintained by Ernest Davis documents, the sentence appears in a slightly different form in the 1970 thesis and in its familiar form in the 1972 book. In 2011 Hector Levesque proposed turning such pairs into a test, the Winograd Schema Challenge, built so that no simple statistical association over text would resolve them; a formal contest was run once, in 2016.

The outcome inverted SHRDLU's premise. The GPT-3 paper reported 88.3 percent zero-shot accuracy on the standard 273-problem set, close to the 90.1 percent of the best fine-tuned system, while flagging possible contamination of that test data. A review titled The Defeat of the Winograd Schema Challenge records that by 2019 fine-tuned transformer language models had passed 90 percent. The authors of WinoGrande, a 44,000-problem version built to strip out exploitable word associations, found the best models of the time reaching 59.4 to 79.1 percent against 94.0 percent for people, and warned that machine commonsense was probably being overestimated. A puzzle chosen to show that language needs a model of the world was largely solved by systems that learned from text alone, and whether that counts as understanding is still disputed.

How to place SHRDLU in AI history

SHRDLU is best read as a demonstration of integration under ideal conditions. It showed that parsing, semantics, deduction, planning and a memory of the conversation could be combined so tightly that a program seemed to grasp its own subject matter, in a domain chosen to be small enough to know completely. That was a real result, and the printed dialogue made it legible to people outside the field, which is why Lighthill singled it out, why it shaped psycholinguistics, and why it anchored the procedural versus declarative argument of the 1970s.

Its limits were equally real and were named at the time, by Lighthill on computational grounds and by Winograd himself on conceptual ones. The program did not learn its knowledge, could not absorb genuinely new concepts, and had no path to the open-endedness of ordinary language. Winograd's own later summary in the oral history is generous in a qualified way: he called SHRDLU a spark plug that got people excited, while believing the direction it stood for would not work. The most accurate placement holds both judgments at once. SHRDLU marks the high point of hand-built language understanding inside a microworld, and the clearest early evidence of how narrow that road was.

References

• Winograd (1970). Procedures as a representation for data in a computer program for understanding natural language, PhD thesis record.
• Winograd (1971). Procedures as a Representation for Data in a Computer Program for Understanding Natural Language, MIT AI Technical Report 235.
• Computer History Museum (1971). MAC TR-84, Procedures as a Representation for Data in a Computer Program for Understanding Natural Language.
• Winograd (1972). Understanding natural language, Cognitive Psychology vol. 3.
• Minsky and Papert (1972). Artificial Intelligence Progress Report, MIT AI Memo 252.
• Lighthill (1973). Artificial Intelligence: A General Survey.
• Sutherland (1973). Some comments on the Lighthill report and on Artificial Intelligence.
• Chilton Computing (1973). Lighthill Report overview.
• Weizenbaum (1966). ELIZA: a computer program for the study of natural language communication between man and machine.
• Nilsson (2009). Progress in Natural Language Processing, The Quest for Artificial Intelligence.
• Stanford Historical Society (2023). Terry Winograd: An Oral History.
• Winograd (2025). Terry Winograd home page.
• Winograd and Flores (1987). On understanding computers and cognition: a new foundation for design, reply to reviews.
• Davis, Morgenstern and Ortiz (2024). The Winograd Schema Challenge.
• Davis (2021). A collection of Winograd schemas.
• Brown et al. (2020). Language Models are Few-Shot Learners.
• Sakaguchi et al. (2019). WinoGrande: An Adversarial Winograd Schema Challenge at Scale.
• Kocijan et al. (2022). The Defeat of the Winograd Schema Challenge.