MYCIN expert system: A consultant built from rules

The event and the narrow claim

In 1976 Elsevier published Edward Shortliffe's Computer-Based Medical Consultations: MYCIN, a book adapted largely from his Stanford doctoral thesis and issued as the second volume in an artificial intelligence series edited by Nils Nilsson ( the book's front matter and copyright page). It described a program that questioned a physician about a patient with a bacterial infection, decided whether treatment was needed, inferred the likely organisms and recommended antibiotics, all from 200 conditional rules supplied by Stanford specialists in infectious disease and clinical pharmacology ( Shortliffe's concluding chapter). It could show the rule behind any question it asked, and experts could change rules without rewriting the program.

The narrow, documented claim concerns architecture and performance on written cases. MYCIN showed that judgmental clinical knowledge could be written as separate if-then rules, kept apart from a generic reasoning procedure, and used to produce advice that outside specialists rated well. A blinded study published in JAMA in 1979 found its antibiotic choices for ten difficult meningitis cases were rated acceptable more often than those of five Stanford faculty specialists ( Yu and colleagues' evaluation in JAMA). What MYCIN did not do is equally on the record: it never worked on hospital wards, and its infectious disease knowledge base was retired in 1978 ( the project's own retrospective). Its weight rests on the template it set and the descendants it produced.

From a drug-interaction monitor to a bedside consultant

MYCIN grew out of a neighbouring project. Stanley Cohen, then chief of clinical pharmacology at Stanford's medical school, ran MEDIPHOR, a system that warned physicians about drug interactions, and had recruited both the computer scientist Bruce Buchanan and a medical student, Edward Shortliffe, who had discovered AI in a computer science course ( the history in Buchanan and Shortliffe's 1984 volume). Discussion turned to a program that would flag poor antibiotic prescriptions. The team saw that such a monitor would first have to be an expert in antimicrobial selection, and that an interactive consultant would avoid linking the hospital's pharmacy, laboratory and bacteriology computers. Cohen recruited Thomas Merigan, chief of infectious diseases, and Merigan's colleague Stanton Axline.

Two early choices shaped everything after. Cordell Green suggested Interlisp, which was then running at the Stanford Research Institute rather than at the university, and the team tried and dropped a semantic network design in favour of rules, because Cohen and Axline found chained rules easier to read and criticize ( the same chapter). When the first grant application went in during October 1973, the collaboration was already about a year and a half old; the name, chosen at Axline's suggestion, is simply the suffix shared by many antimicrobial drugs. The book's acknowledgements record that most computer time came from the Stanford Research Institute under an ARPA contract and that Shortliffe was supported by the NIH Medical Scientist Training Program ( the 1976 front matter).

The lineage ran through DENDRAL. Joshua Lederberg and Edward Feigenbaum had persuaded Buchanan that the ideas behind their chemistry program could serve a medical problem, and the lesson that carried over most directly was the power of production rules ( the 1984 volume's opening chapter). The difference was one of degree that became one of kind: the MYCIN authors judged that no earlier AI program, DENDRAL included, had kept so much domain knowledge so cleanly separated from its inference procedures.

Why antibiotic choice made a good test

The task combined real clinical need with a bounded scope. A physician facing a seriously ill patient usually cannot wait for the laboratory: definitive identification of an organism from a culture commonly takes 24 to 48 hours or more, and sensitivity testing comes later still, so treatment must start on partial evidence such as the Gram stain, the organism's shape, the site of the culture and the patient's history ( the 1984 volume's opening chapter). The authors framed the job as four linked decisions: whether the infection is significant, what the organism probably is, which drugs might work, and which is best for this patient given allergies, age and kidney function.

There was also evidence that help was needed. The same chapter summarizes a 1972 study by Roberts and Visconti of patients admitted to a community hospital, in which expert reviewers judged only 13 percent of systemic antimicrobial use rational and 66 percent clearly irrational ( the 1984 volume's opening chapter). Shortliffe's book presented MYCIN as a way to bring specialist-level advice to non-specialists when an infectious disease consultant was not at hand ( Shortliffe's concluding chapter).

Two hundred rules and a goal stack

By the 1976 account, MYCIN's knowledge lived in 200 rules over 65 clinical parameters, and the program ran on a Digital Equipment Corporation PDP-10 with 256K of core memory ( Shortliffe's concluding chapter). A representative rule, in its English rendering, said that if an organism's identity was not yet known with certainty, its stain was Gram-negative, its shape was a rod and it grew aerobically, then there was strongly suggestive evidence, rated 0.8, that it belonged to the enterobacteriaceae ( Shortliffe's chapter on the consultation system). Other rules excluded certain drugs whenever meningitis was diagnosed, or listed ranked drug choices for a given organism.

The control strategy was goal-directed backward chaining. A single goal rule stated that if any organism required therapy, the program should compile the possible drugs and choose the best; a consultation consisted of creating a record for the patient and trying to satisfy that one rule ( the consultation system chapter). To decide whether an organism needed therapy, MYCIN looked for rules that could conclude whether it was significant, which in turn needed facts such as the culture site, and so on down the chain, asking the physician when no rule could supply a value. Peter Szolovits of MIT summarized the insight: behaviour that might need a flowchart hundreds of pages long came from a few hundred concise rules and a short recursive procedure ( Szolovits's overview of medical AI).

Because every question came from a specific rule, explanation came almost free: typing WHY made MYCIN display the rule it was trying to apply. In the book's sample session, asked why it wanted to know about procedures on the urinary tract, the program replied that it already knew the organism was a Gram-negative rod cultured from blood with an upper genitourinary portal of entry, and that if the patient had had no such procedure and no treated urinary infection, RULE156 would give suggestive evidence, rated 0.6, that it was E. coli ( the book's introduction and sample consultation). Repeated WHY questions climbed the stack of goals, and HOW questions traced how a conclusion had been reached ( the 1984 chapter on lessons). LISP mattered here: the authors chose it partly because it treats programs as data, so rules could be executed and also inspected and edited ( the 1984 volume's opening chapter).

Arithmetic for hunches: certainty factors

Clinicians' rules are rarely certain, and the team judged that a Bayesian treatment would demand conditional probabilities nobody had collected. Shortliffe and Buchanan's 1975 paper in Mathematical Biosciences instead defined a measure of increased belief and a measure of increased disbelief in a hypothesis given evidence, and a certainty factor equal to the first minus the second, ranging from minus one to plus one ( their model of inexact reasoning). When two rules supported the same conclusion, MYCIN combined their factors so that confirmation accumulated without exceeding one: a stored value X updated by new evidence Y became X plus Y times one minus X ( the consultation system chapter). A value of 0.2 or less was treated as unknown, a threshold chosen empirically and, by the authors' later account, mainly to stop the program pursuing weak leads and asking pointless questions ( the 1984 chapter on lessons).

The scheme was widely adopted, largely because it let each rule be written on its own ( Shortliffe's page for the 1976 book). It also hid assumptions. David Heckerman showed in 1985 that the original definitions conflicted with the combining functions, and that any consistent probabilistic reading required evidence to be conditionally independent given both a hypothesis and its negation, and the network of inferences to be a tree, conditions he judged rarely true in practice ( Heckerman's analysis of certainty factors). In 1992 Heckerman and Shortliffe set out the model's limits and argued that belief networks, grounded in probability, overcame many of them ( their review). The MYCIN authors had already found, through sensitivity analysis, that the system did not depend on precise certainty values, and that experts needed no more than four or five grades of evidential strength even when offered a continuous scale ( the 1984 chapter on lessons).

Grading a machine against specialists

MYCIN's builders treated evaluation as part of the research, wary of the anecdotal praise for which both AI and medical computing had been criticized ( the 1984 chapter on evaluation). The first study, run in mid-1974 for Shortliffe's thesis, asked five infectious disease experts to judge the program's handling of fifteen bacteremia cases; they approved its therapy in 72 percent of the 75 evaluations, and the book set a target of at least 90 percent approval before MYCIN would be placed on the wards of Stanford Hospital and the affiliated Veterans Administration hospital in Palo Alto ( the book's chapter on future directions). A second bacteremia study in the mid-1970s, with ten evaluators from Stanford and other centres, again produced roughly 75 percent approval, though some took more than twelve months to return their forms ( the 1984 chapter on evaluation). Two problems were plain: evaluators knew they were grading a computer, and nobody knew what approval rate the experts themselves would earn.

The third study, published in JAMA in September 1979, was designed to address both. A physician unconnected with the project selected ten diagnostically challenging meningitis cases from a county hospital affiliated with Stanford, including tuberculous, fungal, viral and bacterial infections ( the JAMA article). Ten prescribers worked from the same written summaries: MYCIN, the physicians who had actually treated each patient, five Stanford infectious disease faculty, a fellow, a resident and a medical student. Eight specialists at other institutions, each with published work on meningitis, then rated all 100 prescriptions in random order and a standard format, without knowing that one prescriber was a program, for 800 assessments in all ( the study as reprinted in 1984).

MYCIN's prescriptions were rated acceptable in 65 percent of assessments, against 42.5 to 62.5 percent for the five faculty members, whose mean was 55.5 percent; a majority of evaluators accepted MYCIN's choice in seven of ten cases, against a faculty mean of 44 percent ( the study as reprinted in 1984). The program never failed to cover a treatable pathogen. Neither did three of the faculty or the treating physicians, and the treating physicians also matched MYCIN's seven majority approvals, but they often prescribed two or three drugs where one or none would have done. The three cases in which a majority withheld approval were instructive: in each MYCIN prescribed penicillin alone, for meningococcal, group B streptococcal and Listeria meningitis, where many evaluators preferred other drugs or combinations. The blinded design also showed that the faculty fell far short of the 90 percent bar set in 1976, and the authors later conceded that the target had been unrealistic given how much experts disagree ( the 1984 chapter on evaluation). They were candid that ten cases is a small sample, and called for trials in a clinical setting, where acceptability, effect on patient care, cost and legal questions remained open ( the study as reprinted in 1984).

Why the best-rated prescriber never reached the wards

The 1976 plan was concrete: refine the bacteremia knowledge until experts were satisfied, then put terminals on the wards and measure clinical impact ( the book's chapter on future directions). It never happened. The 1984 retrospective states that the ward trial was never undertaken and that the infectious disease knowledge base was laid to rest in 1978, despite the evaluation results ( the 1984 chapter on lessons).

The project's own accounts give several reasons. Funding split along the project's two goals: after three years of NIH support, only the AI research line was funded, by the National Science Foundation, the Office of Naval Research and DARPA, in work that produced EMYCIN, GUIDON and NEOMYCIN, while clinically minded agencies wanted ward-ready programs and imagined that textbook sentences could be transcribed into rules; by 1977 the medical collaborators were moving on ( the 1984 chapter on lessons). The software ran only in Interlisp under the TENEX and TOPS-20 operating systems, on machines that hospital wards did not have, and the team was not motivated to rewrite it. The consultation model was itself a burden: MYCIN usually asked around 50 or 60 questions, physicians did not want to type, and the authors concluded that clinicians would rather telephone a specialist, and that casting a machine as the expert was a hard sell in medicine. Shortliffe had foreseen part of this in 1976, writing that physicians seldom use computers for tasks they believe they can do themselves and that automatic prescription monitors might influence care more than a consultant physicians had to seek out ( Shortliffe's concluding chapter).

MYCIN was not unusual. Surveying the first generation of medical AI programs in 1982, Szolovits noted that none was in current clinical use ( Szolovits's overview of medical AI). The telling contrast is PUFF, an EMYCIN-built interpreter of lung function tests developed with Pacific Medical Center in San Francisco. Its rules were rewritten in BASIC for a PDP-11 in the hospital's laboratory, it ran in batch mode without questioning any physician, and from 1979 it interpreted tests for about ten patients a day, more than 4,000 cases by the time of the report, with about 85 percent of its reports accepted without change ( Aikins and colleagues' PUFF paper). What reached routine use was the MYCIN machinery fitted into an existing workflow, not the MYCIN consultation.

The engine without the medicine

Because MYCIN kept its rules apart from the code that used them, the code could be lifted out. William van Melle's EMYCIN, for Essential MYCIN, was the domain-independent core, and MYCIN itself was reimplemented as one of the consultation programs running on it ( the EMYCIN chapter). Van Melle's 1979 IJCAI paper reported consultation systems built with it in several areas of medicine and one in engineering ( van Melle's IJCAI paper). The payoff was speed: SACON, advising engineers on a structural analysis program, needed about two person-months of expert time, and CLOT, on blood clotting disorders, reached a preliminary 60 rules after roughly ten hours of discussion and ten of entry ( the EMYCIN chapter).

Related research followed. Randall Davis's TEIRESIAS studied how experts could transfer knowledge to the system and introduced rules about rules; William Clancey's GUIDON tried to tutor students from MYCIN's knowledge base and found that teaching needs more knowledge than advising ( the 1984 volume's opening chapter), a finding that led Clancey to reformulate the knowledge base as NEOMYCIN ( the 1984 chapter on lessons). Shortliffe rejoined the work in 1979 to build ONCOCIN for managing cancer treatment protocols ( the 1984 chapter on lessons). Allen Newell, writing the foreword to the 1984 retrospective as a self-described outsider, called MYCIN "the granddaddy of them all" and argued that it, more than DENDRAL, made the new niche evident to the rest of the field, because its search sat in the background and the system could be seen purely as a body of knowledge ( Newell's foreword). He also counted EMYCIN as a significant way of distilling experience into a reusable kernel, and tied the expert-system wave to the entrepreneurial rush into AI and Japan's Fifth Generation plans.

The honest limits

MYCIN's knowledge was shallow by design. Its rules tied observations to conclusions on expert authority rather than any model of microbiology, knowledge Szolovits called phenomenological, and he warned that such programs show a "plateau and cliff effect": excellent on anticipated cases, degrading sharply just outside them ( Szolovits's overview of medical AI). When two rules of thumb conflicted, MYCIN could only adjust a number, where a human expert would look for a deeper explanation ( the same overview).

The evaluation evidence was careful but narrow: the decisive study used ten meningitis cases, worked from written summaries rather than live patients, and measured agreement with experts rather than patient outcomes ( the study as reprinted in 1984). The uncertainty calculus was coherent only under assumptions that rarely held ( Heckerman's analysis of certainty factors). Popular accounts give varying rule counts; the documented figures are 200 rules in 1976 and, later, 450 rules plus about 1,000 facts in tables covering 120 possible organisms ( the 1984 chapter on lessons). Finally, the project's own funding history shows how badly outsiders misjudged the cost of expertise: building a knowledge base was research, not transcription ( the 1984 chapter on lessons).

How to place MYCIN in AI history

MYCIN has no single launch date. Work began around 1972; the book cites the thesis as a 1974 work ( the 1976 front matter), while Shortliffe's later page for the book says the dissertation was completed in October 1975 ( Shortliffe's page for the 1976 book), journal papers appeared from 1973 to 1975, the book in 1976, the knowledge base was retired in 1978, and the JAMA study appeared in 1979. Chronicle uses 1976 because the book is the first complete public description of the whole system in one place; 1979 is when its performance claim entered a clinical journal.

MYCIN sits between DENDRAL and the commercial systems of the 1980s. DENDRAL showed that specialist knowledge could steer search in chemistry; MYCIN pushed search into the background, made the rule base the system, and added explanation and a workable, flawed calculus of uncertainty. Its direct descendants were EMYCIN, PUFF, GUIDON and ONCOCIN, and its certainty factors were eventually displaced by probabilistic belief networks ( Heckerman and Shortliffe's review). Its non-adoption is part of the lesson rather than a footnote: the authors themselves wrote that they had underestimated the gap between performance and acceptability ( the 1984 chapter on lessons). Decades later Shortliffe still argued that black boxes are unacceptable for direct clinical use and pointed to disrupted clinical workflow as a recurring reason such systems went unused ( Shortliffe's 2019 review). Whether today's clinical AI escapes MYCIN's fate is a provisional question, but its questions of explanation, integration and accountability were posed in practice by this program.

References

• Edward H. Shortliffe, American Elsevier (1976). Computer-Based Medical Consultations: MYCIN, front matter and copyright page.
• Edward H. Shortliffe, American Elsevier (1976). Computer-Based Medical Consultations: MYCIN, Chapter 1: Introduction.
• Edward H. Shortliffe, American Elsevier (1976). Computer-Based Medical Consultations: MYCIN, Chapter 6: Future Directions for MYCIN.
• Edward H. Shortliffe, American Elsevier (1976). Computer-Based Medical Consultations: MYCIN, Chapter 7: Conclusion.
• Edward H. Shortliffe, Columbia University DBMI (1976). Computer-Based Medical Consultations: MYCIN, book page.
• Edward H. Shortliffe and Bruce G. Buchanan, Mathematical Biosciences (1975). A Model of Inexact Reasoning in Medicine, as reprinted in 1984.
• Victor L. Yu, Lawrence M. Fagan, Sharon M. Wraith and colleagues, JAMA (1979). Antimicrobial Selection by a Computer: A Blinded Evaluation by Infectious Diseases Experts.
• William van Melle, IJCAI (1979). A Domain-Independent Production-Rule System for Consultation Programs.
• Peter Szolovits, Westview Press (1982). Artificial Intelligence and Medicine.
• Janice S. Aikins, John C. Kunz, Edward H. Shortliffe and Robert J. Fallat, Computers and Biomedical Research (1983). PUFF: An Expert System for Interpretation of Pulmonary Function Data.
• Bruce G. Buchanan and Edward H. Shortliffe, Addison-Wesley (1984). The Context of the MYCIN Experiments.
• Edward H. Shortliffe, Addison-Wesley (1984). Details of the Consultation System.
• William van Melle, Edward H. Shortliffe and Bruce G. Buchanan, Addison-Wesley (1984). EMYCIN: A Knowledge Engineer's Tool for Constructing Rule-Based Expert Systems.
• Bruce G. Buchanan and Edward H. Shortliffe, Addison-Wesley (1984). The Problem of Evaluation.
• Victor L. Yu and colleagues, Addison-Wesley (1984). An Evaluation of MYCIN's Advice.
• Bruce G. Buchanan and Edward H. Shortliffe, Addison-Wesley (1984). Major Lessons from This Work.
• Allen Newell, Addison-Wesley (1984). Foreword to Rule-Based Expert Systems: The MYCIN Experiments.
• David Heckerman, Uncertainty in Artificial Intelligence (1985). Probabilistic Interpretations for MYCIN's Certainty Factors.
• David Heckerman and Edward H. Shortliffe, Artificial Intelligence in Medicine (1992). From Certainty Factors to Belief Networks.
• Edward H. Shortliffe, Yearbook of Medical Informatics (2019). Artificial Intelligence in Medicine: Weighing the Accomplishments, Hype, and Promise.