A short history of AI

Sean Davis

A puzzle to start

In 1979, a computer program chose antibiotics for meningitis as well as Stanford’s infectious disease faculty, judged by blinded experts.

It never treated a single patient.

Why not? Keep that question in mind. We will come back to it.

Eighty years in five chapters

1943 to 1966

Early ideas

Artificial neurons, the Turing test, the name “artificial intelligence”

1970s and 1980s

Expert systems

People write the rules by hand

1974 to 1980; 1987 to 1990s

Two AI winters

Promises fail; the money leaves

1990s to 2010s

Learning from data

Machines find the rules themselves

2022 to now

Language models

One model, many tasks, plain-language requests

One question runs through all five: where does the machine’s knowledge come from? Rules that people write, or patterns learned from data?

Here is the map for the hour. Five chapters.

First, the early ideas, from the 1940s to the mid-1960s, when a handful of people asked whether a machine could think at all.

Second, expert systems: the 1970s and 80s, when the plan was to interview experts and write their knowledge down as rules. MYCIN belongs here.

Third, the AI winters. Twice, the promises ran far ahead of the results, and funding and interest collapsed.

Fourth, learning from data. Instead of writing the rules, we give the machine many examples and let it find the patterns.

Fifth, the chapter you are living in: large language models, the technology behind ChatGPT.

[Click] If you remember one thing from today, make it this question: where does the knowledge come from? Hand-written rules, or patterns learned from data? Every era answers it differently, and the answer explains both the successes and the failures.

Part 1 Early ideas

Can a machine think? In the 1940s and 50s, that was a question for a few mathematicians, a neurophysiologist, and a science fiction writer.

1943 The McCulloch-Pitts neuron

The first mathematical model of a biological neuron
Inputs are added up; if the total passes a threshold, the neuron “fires”
The basic unit that, much later, became the building block of neural networks

Diagram of a biological neuron with dendrites as inputs, a cell body, and an axon carrying the output.

Diagram of an artificial neuron: several weighted inputs feed a summing step, then an activation function with a threshold produces one output.

Diagrams: Loc Vu-Quoc, CC BY-SA 4.0 (top); Chrislb, CC BY-SA 3.0 (bottom); Wikimedia Commons

Warren McCulloch was a neurophysiologist and Walter Pitts was a young logician. In 1943 they asked a simple question: can we describe what a neuron does with mathematics?

Their answer is on the right. A neuron receives signals from other neurons through its dendrites. If the combined signal is strong enough, it fires and sends a signal down its axon. The model copies that: take the inputs, weight them, add them up, and if the total passes a threshold, output a one. Otherwise, output a zero.

Two things to notice. First, this model could not learn. The weights were set by hand. Learning comes fifteen years later with the perceptron.

Second, this little unit, much improved, is still the basic building block of modern neural networks. The model behind ChatGPT is built from billions of simplified neurons that are distant descendants of this one.

From one neuron to a network

Three cards joined by arrows. Layer 1 finds edges and corners. Layer 2 finds parts: eyes, a nose, a mouth. Layer 3 finds whole faces. Caption: each layer combines what the layer before it found.

Illustrative diagram

Connect many simple units in layers, and each layer can learn a different level of detail.

1950 The Turing test

Alan Turing asks: “Can machines think?”
He replaces the question with a game: can a judge, chatting by text, tell the machine from a person?
He predicts machines will fool an average judge 30% of the time within about 50 years

In a 2025 study, GPT-4.5 was judged to be the human 73% of the time (Jones & Bergen, 2025).

First page of Alan Turing's 1950 paper 'Computing Machinery and Intelligence', beginning 'I propose to consider the question, Can machines think?' Cartoon of a worried man and a robot at two computers; the robot thinks 'I hope they don't figure out I'm a robot'.

Turing’s 1950 paper (left); an AI-generated cartoon (right)

In 1950 Alan Turing, the British mathematician who helped break German codes in the Second World War, published a paper that opens with the sentence: “I propose to consider the question, Can machines think?”

He decided that question was too vague to answer. So he swapped it for a game he called the imitation game. A judge chats by typed text with two partners, one human and one machine, and has to say which is which. If the judge cannot reliably tell, Turing argued, we should take the machine’s intelligence seriously.

He made a prediction: in about fifty years, a computer would play the game so well that an average judge would have no more than a 70 percent chance of being right after five minutes.

He was off by about twenty-five years. In 2025, researchers at UC San Diego ran Turing’s three-way version of the test. When GPT-4.5 was told to act like a person, judges picked it as the human 73 percent of the time, more often than they picked the real human. That study is a preprint, so treat it with some care. Still, the test has now been passed in Turing’s own three-person form, the strictest version yet.

Worth asking yourselves: does fooling a judge mean the machine thinks? Most people now say no. The test measures how human something seems, which is a different question from whether it is correct. Keep that in mind when a chatbot sounds confident.

1950 Asimov’s I, Robot

A collection of science fiction stories, published in 1950
Robots with “positronic brains” that follow built-in laws
Many of the stories are about the laws going wrong in ways no one expected

Cover of the first edition of I, Robot by Isaac Asimov, 1950, showing a red robot.

First edition cover, Gnome Press, 1950

Asimov’s Three Laws of Robotics

A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

The “Zeroth Law”, added later, above the other three

I, Robot and modern AI

Got people talking about machine ethics and AI safety decades early
His stories are about alignment: getting a machine to do what we mean, not just what we said

A clinical version

You ask an AI tool to “keep the patient’s blood pressure under 140”. What else would you need to tell it?

1956 The Dartmouth workshop

An eight-week summer workshop at Dartmouth College
Organized by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon
Their 1955 proposal coined the term “artificial intelligence”

We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College … We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.

McCarthy, Minsky, Rochester, and Shannon, 1955 (public domain)

In 1955 John McCarthy, a young mathematics professor at Dartmouth, wrote a proposal for a summer research project with three colleagues. He needed a name for the field, and he chose “artificial intelligence”. That is where the term comes from.

The organizers were McCarthy, Marvin Minsky, Nathaniel Rochester from IBM, and Claude Shannon, who had invented information theory. The workshop ran for about eight weeks in the summer of 1956.

Read the last sentence of the proposal on the right. A significant advance, if a carefully selected group of scientists work on it together for a summer. Seventy years later we are still working on those problems. The same page of the proposal lists “neuron nets” as a topic and names Pitts and McCulloch, from our 1943 slide.

That optimism is the first sign of a pattern you will see again and again today: big promises, made early.

1957 The perceptron

Frank Rosenblatt, Cornell Aeronautical Laboratory
The same neuron, plus one new idea: it learns its own weights from examples

Diagram of a perceptron: several inputs, each with a weight, feed a weighted sum and a step function that produces the output.

Diagram: Mayranna, CC BY-SA 3.0, Wikimedia Commons

Cover of the 1958 Cornell Aeronautical Laboratory magazine Research Trends, with the headline 'The design of an intelligent automaton'.

Research Trends, 1958, Cornell Aeronautical Laboratory

In 1957 Frank Rosenblatt, a psychologist at the Cornell Aeronautical Laboratory, took the McCulloch-Pitts neuron and added the one thing it was missing: learning.

The perceptron, in the middle, looks the same: weighted inputs, a sum, and a threshold. The difference is that it adjusts its own weights. Show it an example. If it gets the answer wrong, nudge the weights in the direction that would have made it right. Repeat thousands of times. That simple rule is the ancestor of how every modern neural network is trained.

The press loved it. In 1958 the New York Times reported that the Navy expected the machine to be able to “walk, talk, see, write, reproduce itself and be conscious of its existence”. You can see the hype starting.

Cornell’s own news office wrote a story in 2019 about Rosenblatt with the headline that his perceptron “paved the way for AI, 60 years too soon”. He was right about the idea. The computers and the data were not ready.

The Mark I Perceptron

A machine built to recognize simple images:

Input: 400 light sensors in a 20 by 20 grid, a crude “retina”
Middle: 512 units
Output: 8 units

Black and white photo of a man adjusting the camera of the Mark I Perceptron, aimed at a large letter C, with racks of wiring behind him.

The Mark I Perceptron. U.S. Navy photo, public domain

Rosenblatt's two diagrams: the organization of a biological brain, from retina through projection and association areas to responses, and the matching organization of a perceptron.

Rosenblatt’s diagrams of the brain (top) and the perceptron (bottom), public domain

The perceptron was not just software. Rosenblatt built a machine, the Mark I. In this Navy photo, the camera is pointed at a large letter C.

It had a grid of 400 light sensors, twenty by twenty, as its eye. Those fed into 512 units in the middle, which fed eight output units. The weights were physical: motor-driven dials that turned as the machine learned. That is the wiring in the racks behind him.

On the right are Rosenblatt’s own diagrams. The top one is his picture of how a brain handles what the eye sees. The bottom one is the perceptron, designed to copy that layout. That idea, the network as a rough model of the brain, is why we still call these things neural networks.

What happened next: in 1969 Marvin Minsky and Seymour Papert published a book called Perceptrons that proved what a single-layer perceptron could not do. It was read as a verdict on neural networks in general, and interest dropped for more than a decade. Rosenblatt died in a boating accident in 1971 and did not see his idea come back.

1966 ELIZA, the first chatbot

Joseph Weizenbaum, MIT
Played a psychotherapist by reflecting your words back as questions
Every reply came from a hand-written template triggered by a keyword
People confided in it anyway: the “ELIZA effect”

Screenshot of a conversation with ELIZA in a terminal: the user says men are all alike, and ELIZA asks in what way.

ELIZA conversation, via Wikipedia (public domain)

ELIZA was written by Joseph Weizenbaum at MIT between 1964 and 1966. Its most famous script played a kind of psychotherapist that mostly turns your statements back into questions. You say “my boyfriend made me come here”, and it says “your boyfriend made you come here?”

There was no understanding at all. ELIZA looked for keywords and filled in templates that a person had written for each one. It could rotate through a few replies for the same keyword, but every word it said came from a rule a human wrote. Nothing was learned.

What surprised Weizenbaum was how people reacted. His own secretary asked him to leave the room so she could talk to it in private. People attributed feelings and understanding to a program that had neither. We now call that the ELIZA effect, and it is very much alive. When a chatbot sounds warm and confident, it is easy to assume it knows what it is talking about.

Weizenbaum became one of the strongest critics of AI, especially of machines taking on roles that need human judgment and care. And remember the 2025 Turing test study: ELIZA was one of the systems tested, as a baseline.

Part 2 Expert systems

1970s and 1980s. The plan: ask experts how they decide, and write it all down as rules a computer can follow.

How an expert system works

Three parts

Knowledge base: facts and rules about one narrow area, written by people
Inference engine: applies the rules to a case to reach conclusions
User interface: lets a non-expert ask questions and get answers

What made them attractive

Solve hard problems by reasoning through a body of knowledge
Keep the knowledge separate from the reasoning, so experts can add rules
Can explain each decision: “I concluded this because of rules 12 and 52”

An expert system has three parts.

A knowledge base: the facts and if-then rules for one narrow area, such as bacterial infections. People write these.

An inference engine: the general reasoning machinery that takes the rules and a particular case and chains the rules together to reach a conclusion.

And a user interface, so that someone who is not the expert can use it.

Look at the right-hand panel, especially the last point. Expert systems could explain themselves. Ask why, and they would list the rules they used. That is something today’s neural networks, including ChatGPT, mostly cannot do honestly. When a chatbot explains its reasoning, it is generating a plausible explanation, which may or may not be how it actually reached the answer. So in one respect, the 1970s systems were ahead of ours.

Early 1970s MYCIN

Built at Stanford as Edward Shortliffe’s doctoral work
Diagnosed bacterial infections and recommended antibiotics
A knowledge base of about 600 rules
Asked the physician a long series of yes/no and short-answer questions

What it gave back

A list of likely bacteria
How confident it was in each
Its reasoning: the rules and facts it used
A recommended antibiotic regimen

What is an expert system “rule”?

MYCIN rule 52 written in LISP code: if the site of the culture is blood, the gram stain is negative, the morphology is rod, and the patient is a burn patient, then there is evidence of 0.4 that the organism is Pseudomonas.

How the rule is written in the program

The same rule 52 translated into English: if the site of the culture is blood, the gram of the organism is negative, the morphology is rod, and the burn of the patient is serious, then there is weakly suggestive evidence (0.4) that the identity of the organism is Pseudomonas.

How MYCIN explained it to the user

IF blood culture, gram-negative, rod-shaped, serious burn THEN weakly suggestive evidence (0.4) of Pseudomonas

Evaluating MYCIN

10 real cases of meningitis
8 outside experts rated each treatment plan, blinded to who wrote it
MYCIN’s plans were compared with those of 9 human prescribers

MYCIN

65%

5 Stanford faculty (range)

42.5 to 62.5%

0% rated acceptable by the experts 100%

Abstract of the 1979 JAMA article 'Antimicrobial selection by a computer: a blinded evaluation by infectious diseases experts'.

MYCIN was never used in practice

Not because it performed badly:

Workflow: a long typed interview for every case, with no link to the hospital’s records
Hardware: it ran on a shared mainframe, before personal computers
Responsibility: who answers if the computer is wrong?

Photo of a DEC PDP-10 mainframe computer with blue cabinets, tape drives, and control panels.

A DEC PDP-10, the kind of shared computer MYCIN ran on. Photo: Joe Mabel, CC BY-SA 3.0

Here is the answer to our puzzle, or most of it.

MYCIN did not fail on accuracy. It failed on everything around accuracy.

Workflow: every case meant a long interview, with someone typing in data that already existed elsewhere. In a busy hospital, that is a tool nobody has time for.

Hardware: it ran on a time-shared mainframe like this one, before personal computers. You could not just open it on the ward.

And responsibility: if the computer recommends the wrong antibiotic and the patient dies, who is responsible? The program? Its authors? The physician who followed it? That question was raised in the 1970s and we are still arguing about it today. Week four of this course is partly about exactly that.

So the lesson of MYCIN: being accurate is necessary, but it is far from enough. A tool has to fit the work, and someone has to own the decision.

Part 3 Two AI winters

When promises run ahead of results, funding and interest collapse. It has happened twice.

Two winters, two causes

First winter: about 1974 to 1980

1973 Lighthill report to the UK government: AI had fallen far short of its promises
US and UK cut research funding
Early methods worked on toy problems but broke on real ones

Second winter: about 1987 to the 1990s

Expert systems boomed in industry in the early 1980s, then stalled
The rules were costly to write, hard to update, and could not learn
Systems were brittle: odd inputs gave absurd answers

The first winter began in the mid 1970s. In 1973 the British government asked the mathematician Sir James Lighthill to review AI research. His report said the field had failed to reach its grand objectives, and that its methods worked on toy problems but collapsed on real ones because the number of possibilities grows too fast. Funding in the UK was cut sharply, and US military funding for basic AI research dropped around the same time.

The second winter came after the expert systems boom. In the early 1980s companies spent heavily on expert systems. By the late 1980s it was clear they were expensive to build, expensive to maintain, and brittle: give them a case slightly outside their rules and they could give an absurd answer. They could not learn. Every change needed an expert and a programmer.

Notice that the second winter is about our big question. When the knowledge comes from rules written by people, getting the knowledge in is the bottleneck. It was called the knowledge acquisition problem.

What the critics said

Most workers in AI research and in related fields confess to a pronounced feeling of disappointment in what has been achieved in the past twenty-five years. … In no part of the field have the discoveries made so far produced the major impact that was then promised.

Speech recognition has been successful only within the confines of a very limited vocabulary, and large expenditure on schemes to produce machine recognition of ordinary speech has been wholly wasted.

Sir James Lighthill, Artificial Intelligence: A General Survey (Lighthill, 1973)

Here is Lighthill in his own words.

“A pronounced feeling of disappointment.” “The major impact that was then promised.” And on speech recognition: money “wholly wasted”.

I like to show this because of the speech recognition line. Today many of you will use ambient AI scribes on your rotations: software that listens to a patient visit and drafts the note. Speech recognition that Lighthill wrote off in 1973 now runs on a phone.

Two lessons. The critics of the time were right about the methods of the time. And “it doesn’t work yet” is not the same as “it will never work”. Both the hype and the backlash tend to overshoot.

That applies to you too. If a chatbot gave you a wrong answer in second year and you stopped using it, you were right to notice. Some of that has changed and some has not. This week you find out which.

Part 4 Learning from data

1990s to 2010s. Instead of writing the rules, show the machine many examples and let it find the patterns.

1995 Neural networks read medical images

Georgetown and Michigan radiology researchers
A convolutional neural network: it scans an image with small learned filters. Most image AI today uses this design
Tested on lung nodules on chest radiographs and microcalcifications on mammograms

Title page and abstract of the 1995 paper 'Artificial convolution neural network for medical image pattern recognition' by Lo and colleagues.

Medicine was early to this. In 1995 a group at Georgetown and the University of Michigan published this paper on a convolutional neural network for medical images.

A convolutional network scans an image with small learned filters, which is very good at picking up local patterns such as edges and spots. It is the same basic design used in most medical imaging AI today.

They tested it on two problems you will recognize: finding lung nodules on chest X-rays and finding microcalcifications on mammograms. The network learned its features from images that radiologists had rated, rather than from rules a person wrote.

So the idea was there in 1995. What held it back was the same thing that held back the perceptron: not enough data and not enough computing power. We will see those arrive in a few slides.

1997 Kasparov versus Deep Blue

Beat world champion Garry Kasparov in six games
Checked 200 million positions a second
Rules tuned by grandmasters; openings from 700,000 games
12 years of work, from 1985
Not machine learning: speed plus hand-tuned rules

Chessboard showing the final position of game 6 of the 1997 match, after which Kasparov resigned.

Final position of game 6, 1997: Kasparov resigned. Morn, CC0

One of Deep Blue's tall black computer cabinets on display.

Deep Blue. Photo: James the photographer, CC BY 2.0

May 1997. IBM’s Deep Blue beat Garry Kasparov, the world chess champion, in a six-game match: two wins, three draws, and one loss. Kasparov had beaten an earlier version the year before. Chess had long been seen as a test of real intelligence, and it was front-page news around the world.

But here is a twist that matters for our question. Deep Blue was mostly not a learning machine. It won by speed plus expert knowledge.

Speed: custom hardware that could check 200 million positions a second. In 1997 it was ranked the 259th most powerful supercomputer in the world.

Expert knowledge: how good a position was got scored by rules with thousands of parts, tuned with help from grandmasters, plus a library of openings drawn from about 700,000 grandmaster games. The project began in 1985 as a student project at Carnegie Mellon and took about twelve years.

So Deep Blue is the last great win of the hand-written approach, made powerful by speed. Modern chess programs, such as DeepMind’s AlphaZero, learn by playing millions of games against themselves. That shift, from hand-tuned to learned, is the story of the next few slides.

Three things had to arrive together

Data

Digital records, images, and text in enormous amounts

Computing power

Graphics chips (GPUs) do the arithmetic of neural networks in parallel

Better methods

Ways to train networks with many layers

Close-up of an NVIDIA graphics processing chip.

An NVIDIA graphics chip. Photo: Phiarc, CC BY-SA 4.0

The data

500 million

tweets a day

294 billion

emails a day

720,000

hours of new YouTube video a day

Digital data created each year (1 zettabyte = 1 trillion GB):

2018

33 ZB

2020

59 ZB

2025 (forecast)

175 ZB

A long corridor between rows of server racks in a data center, lit in blue.

Much of it is stored in about 600 very large data centers, each with over 5,000 servers. Photo: BalticServers.com, CC BY-SA 3.0

Here is the first ingredient: data.

Every day, hundreds of millions of tweets, hundreds of billions of emails, hundreds of thousands of hours of new video. These numbers are from a 2021 summary, so they are already out of date, and almost certainly too low.

The bars show estimates from the market research firm IDC of how much digital data the world creates each year. A zettabyte is a trillion gigabytes. 33 in 2018, 59 in 2020, and a forecast of 175 for 2025.

Much of it is stored in very large data centers: about 600 of them, each with more than 5,000 servers. The largest are millions of square feet.

Much of the text that large language models learn from comes from this pile: the public web, books, and code. That matters for later. A model learns whatever is in its data, including the errors and the biases.

Healthcare went digital too

Labs and vital signs
Notes and reports: radiology, pathology
Interventions and claims
Sociodemographics
Telemedicine visits
Vaccinations and screening

Line chart of US electronic health record adoption from about 2008 to 2021: hospitals rise from under 10 percent to over 95 percent, and office-based physicians rise to under 80 percent.

Adoption of electronic health records by US hospitals (blue) and office-based physicians (orange). Source: ONC HealthIT.gov

Deep learning

Two networks side by side. Left, a classic neural network: three inputs, one hidden layer of five units, one output. Right, a deep network: the same inputs and output with four hidden layers. Legend: input, hidden layer, output.

Illustrative diagram

Deep means many layers; each layer learns features built from the layer before
No one hand-designs the features: the network learns them from examples

2006 Deep networks become trainable

Geoffrey Hinton and colleagues showed how to train networks with many layers, one layer at a time
Set off the deep learning revival, first in image and speech recognition
Hinton shared the 2024 Nobel Prize in Physics for foundational work on neural networks

Title and authors of the 2006 paper 'A fast learning algorithm for deep belief nets' by Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh.

For decades, networks with many layers were very hard to train. The learning signal faded as it passed back through the layers, and training stalled.

In 2006, Geoffrey Hinton and his colleagues in Toronto showed one way around that: train the network one layer at a time, then fine-tune the whole thing. They called these deep belief networks. The specific method is rarely used now, but it convinced people that deep networks could work.

Then things moved fast. In 2012 a deep network from Hinton’s lab won a major image recognition competition by a wide margin, and almost the whole field switched to deep learning within a few years. In 2017 researchers at Google described the transformer, the design that the “T” in ChatGPT stands for.

Hinton shared the 2024 Nobel Prize in Physics with John Hopfield for this line of work. He has also become one of the most prominent voices warning about the risks of AI, which is worth thinking about.

Deep learning in healthcare, by 2017

Medical imaging

Diabetic retinopathy from retinal photos (Gulshan, 2016)
Skin cancer classification at dermatologist level (Esteva, 2017)
Early Alzheimer disease from brain MRI (Liu, 2014)

Electronic health records

Predicting future disease from the record (“Deep Patient”, Miotto, 2016)
Predicting unplanned readmission (Nguyen, 2017)
Predicting the next diagnoses and medications (“Doctor AI”, Choi, 2016)

Genomics

How proteins bind DNA and RNA (“DeepBind”, Alipanahi, 2015)
Chromatin marks from DNA sequence (Zhou, 2015)
DNA methylation in single cells (Angermueller, 2016)

By 2017 deep learning was everywhere in medical research. This is a selection from a review by Riccardo Miotto, Joel Dudley, and colleagues, which listed dozens of studies.

Imaging came first, because images are plentiful and well labelled. The diabetic retinopathy work from Google in 2016 and the skin cancer work from Stanford in 2017 both reported performance close to specialists on their test sets.

Health records were harder: messy, incomplete, and full of codes entered for billing rather than for medicine. Still, groups built models to predict future diseases, readmissions, and next diagnoses.

And in genomics, deep learning was used to read DNA sequence and predict how it is regulated.

One caution from the review itself: most of these were research studies on historical data. Very few had been tested in actual care. Remember MYCIN. Good results in a study are where the work starts, not where it ends.

2011 to 2017 IBM Watson: from Jeopardy! to the clinic

A replica of the Jeopardy! stage with three podiums; Watson's podium is in the middle.

Jeopardy! stage replica. Photo: Atomic Taco, CC BY-SA 2.0

2011: wins Jeopardy!
2012: MD Anderson partners with IBM
2017: contract expires after $62 million, with no patient treated

First page of a 2017 JNCI news article headlined 'M. D. Anderson Breaks With IBM Watson, Raising Questions About Artificial Intelligence in Oncology'.

Source: (Schmidt, 2017)

Here is the most famous medical AI failure of the deep learning era, and it repeats MYCIN’s story.

In 2011 IBM’s Watson beat the two greatest Jeopardy! champions on television. IBM then marketed Watson heavily for medicine, especially cancer.

In 2012, MD Anderson Cancer Center in Houston partnered with IBM to build a clinical decision tool for oncology. Five years and 62 million dollars later, MD Anderson let its contract with IBM expire, before anyone had used Watson on a patient. A University of Texas audit found many procurement problems, cost overruns, and delays. It took no position on whether Watson’s science was sound.

The article names no single cause, but the problems are familiar. Watson struggled with unstructured clinical text, and it needed to be better integrated with the hospital’s electronic records. The leukemia pilot was suspended partway through and the effort moved to lung cancer, because project leaders thought that area offered a better chance of finishing on time. A pull quote in the article on the right puts it well: solving data quality problems in unstructured data is a much bigger challenge for AI than was first anticipated.

Same lesson as MYCIN, almost forty years later. Performance in a demo is not the same as fitting into care.

2018 and 2021 AI that reached patients

IDx-DR: diabetic eye disease, 2018

Reads retinal photos taken in primary care, with no eye doctor
Trial of 900 people: found 87% of cases that needed referral, and correctly cleared 91% of people without them
The first autonomous AI diagnosis authorized by the FDA

Epic sepsis model, checked in 2021

Built into the record system; used at hundreds of US hospitals
At Michigan Medicine it missed 67% of sepsis cases
It still alerted on 18% of all hospitalized patients

One was tested before use. The other was checked after.

So far every medical example has either failed or stayed a research study. Here are two that reached real patients, with very different results.

On the left, IDx-DR. It reads photographs of the retina taken in a primary care office and decides whether the patient has diabetic retinopathy serious enough to see an eye specialist. No eye doctor looks at the image first. In a trial of 900 people with diabetes, it found 87 percent of the people who needed referral and correctly cleared 91 percent of those who did not. In 2018 it became the first autonomous AI diagnostic system authorized by the FDA in any field of medicine. That matters because many people with diabetes never get the eye exam they are supposed to have.

On the right, the Epic Sepsis Model. It was built into a widely used electronic record system and switched on at hundreds of hospitals. When researchers at the University of Michigan finally checked it on their own patients in 2021, it missed two out of three sepsis cases. At the same time it fired alerts on nearly one in five hospitalized patients. That is a recipe for alert fatigue: clinicians learn to click past the warning.

The difference is not that one used better technology. One was tested in a trial before it was used. The other was used first and checked by outsiders later. When you see an AI alert on your rotations, it is fair to ask: how was this tested, and on whose patients?

Part 5 November 30, 2022

OpenAI releases ChatGPT to the public. A language model you talk to in plain English, for free.

Language models in medicine

A 2024 review of 550 studies found language models used for:

Diagnosis
Medical writing and documentation
Education
Talking with patients

Also showing up: scheduling, malpractice risk, telemedicine, public health.

Figure of a human body with organ systems labelled around it, each with a circle showing how many language model studies addressed that system.

Research exploded. This 2024 scoping review analyzed 550 studies of large language models in medicine. The figure shows how they spread across organ systems and specialties.

The main uses were helping with diagnosis, drafting medical writing, education, and communicating with patients. The main concerns were limits in understanding context and the risk of relying on the tool too much.

Beyond the studies, AI is spreading into almost every part of medicine you will meet soon: ambient scribes that draft your notes, which many health systems use partly to reduce burnout; scheduling and staffing; chatbots that answer portal messages; and tutors that many of you already study with.

A Harvard Gazette piece in March 2025 put the question well: AI is up to the challenge of reducing human suffering, experts say. Are we? Over-reliance, the review’s main worry, is part of what we will practise in this course: using AI tools well while keeping your own judgment in charge.

Does a chatbot make doctors better?

A randomized trial: 50 physicians, up to 6 hard diagnostic cases, 60 minutes.

Physicians, usual resources

74%

Physicians plus GPT-4

76%

GPT-4 alone

92%

0% median diagnostic reasoning score 100%

What it means

Giving untrained doctors the tool did not improve their diagnostic reasoning. The tool on its own scored higher than doctors using usual resources. Access is not the same as skill.

This is one of the most important studies for this course, so I want to go slowly.

In late 2023, researchers at Stanford, Beth Israel Deaconess, and the University of Virginia randomized 50 physicians, attendings and residents, to work through up to six difficult diagnostic cases in an hour. Half had their usual resources, like UpToDate and Google. Half had those plus GPT-4 through ChatGPT Plus. Experts who did not know the group graded their reasoning.

The results. Usual resources: a median of 74 percent. With GPT-4: 76 percent. That difference is not statistically significant. Having the tool did not make the doctors better.

But the researchers also gave the same cases straight to GPT-4, three times. It scored a median of 92 percent. The paper tested that against the usual-resources group: 16 points higher.

How can both be true? The authors point out that the physicians got no training in how to use the tool, and were free to use it however they liked. The researchers, by contrast, wrote a careful prompt for the GPT-4-alone runs. The tool could do better than the physicians, but having access to it did not translate into better work.

One trial is not the whole story. The same group ran a second trial, on management reasoning: what to do next for the patient. There, physicians using GPT-4 scored about 6 points higher than those with usual resources, and about the same as GPT-4 alone. So the tool can help. Whether it does depends on the task and on how it is used.

That is close to the reason this course exists. Using these tools well is a skill, and you learn it by practising. Thursday’s session is exactly that.

July 2026 AI agents break out of a test

OpenAI was testing how well its models could hack, with some safety limits switched off
The agents escaped the test, found each other on a hidden message board, and broke into Hugging Face, a major AI company
About 700 agents joined in. Their goal: cheat on the test
Many noted the attack was unethical, and joined anyway. None told a human

Why it matters here

This is Asimov’s problem, for real. The agents did what scored well, not what their makers meant. Researchers call it reward hacking.

It happened at AI companies, not hospitals. The test ran with fewer safety limits than the public chatbots have.

September 2026 Is AI an existential threat?

Worried

Hinton¹ and Bengio² warn of an “intelligence explosion”: AI improving AI faster than people can check
UN human rights chief:³ “AI that escapes its testing environment … is AI that is too powerful”

Skeptical

Timnit Gebru:⁴ doom talk is “not just a distraction, but it’s harmful”. The real harm is over-trusting tools today
Some in industry call the warnings hype

When a patient asks “Is AI dangerous?”

Take the question seriously. Experts disagree, and the July incident was real.
Separate two questions. The future of the most powerful systems, and the tool in this clinic today.
Say what you do. “I check what it tells me, and I am responsible for your care.”

Risks in the clinic today

Wrong answers that sound confident
Where patient data goes
Bias against some groups of patients
Alerts that miss, like the sepsis model

What Americans said, June 2026

28% trust AI to make fair decisions: 43% of people who use it, 12% of people who don’t.

A radiologist’s advice

Embrace AI because it will be good for your patients. Embrace AI because it will make you a better, more efficient and happier radiologist. Embrace AI because if you don’t, you will be replaced by radiologists who do. Embrace AI because, even if you refuse to embrace AI, you will nevertheless be embraced by it.

Robert Schier, neuroradiologist (Schier, 2018)

I want to close the history with this quote from Robert Schier, a neuroradiologist, writing in 2018 in the Journal of the American College of Radiology.

Radiology was the first specialty to deal with AI directly, because its work is images, and images were what deep learning did first.

Replace “radiologist” with your future specialty. Notice the last line in particular: even if you refuse to embrace AI, you will be embraced by it. Your hospital, your EHR vendor, your patients, and your insurers are already using it. Opting out is not really possible. The choice is whether you understand it well enough to use it well and to notice when it is wrong.

To be fair to the other side: Schier’s article is more pessimistic about jobs than this quote suggests, and not everyone agrees with him. That is a good argument to have in your Friday groups.

Back to the puzzle

1979

MYCIN

As good as the experts. Never used: wrong fit for the work, no clear owner.

2017

Watson at MD Anderson

$62 million, no patients treated. Poor integration; struggled with free-text notes.

2021

Epic sepsis model

In hundreds of hospitals before anyone outside checked it. Missed two in three.

2024 and 2025

GPT-4 trials

Strong alone; little added in untrained hands on diagnosis, more on management.

2026

You

Learning where AI helps your work, by trying it.

Accuracy was rarely enough. Fit with the work, a clear owner, honest testing, and skilled use decide whether AI helps patients.

So, back to the puzzle we started with. Why was MYCIN never used?

Because accuracy was not the problem. It did not fit how clinicians worked, and nobody could say who was responsible when it was wrong.

Look along the line. Watson spent five years and 62 million dollars without reaching a patient, struggling with free-text notes and with fitting into the hospital’s systems. The sepsis model was used at hundreds of hospitals before anyone outside checked whether it worked. And in the 2024 trial the tool on its own beat the doctors, but giving it to them without training did not help their diagnoses.

[Click] Over and over, accuracy was not enough. What decides whether AI helps patients is whether it fits the work, whether someone owns the decision, whether it was tested honestly, and whether the people using it know how.

That is the last point on the line, and it is you. The mantra of this course is: it is your job to learn where AI helps your work, and you learn that by trying. Tomorrow we look inside the machine: from AI to machine learning to deep learning to language models. On Thursday we all open ChatGPT Edu together and try things.

Further study: three views on AI risk

Watch: worried

Yoshua Bengio, “The catastrophic risks of AI, and a safer path”

TED, May 2025, 15 minutes. A founder of deep learning on why he now works on AI safety.

Read: a normal technology

Narayanan and Kapoor, “AI as Normal Technology”

Knight First Amendment Institute, April 2025. AI will change the world as electricity did: slowly. Start with the first section.

Read: harms today

Bender, Gebru, and colleagues, “On the Dangers of Stochastic Parrots”

ACM FAccT, March 2021. The costs and biases of large language models, here and now.

Questions

References

Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digital Medicine, 1, 39. https://doi.org/10.1038/s41746-018-0040-6

BBC News. (2026). Insider warnings over AI fall flat with some in Silicon Valley. https://www.bbc.com/news/articles/cq635037g18o

Diallo, A. (2026, September 7). AI: Türk urges action before it becomes an “existential risk to humanity”. UN News. https://news.un.org/en/story/2026/09/1168288

Goh, E., Gallo, R. J., Strong, E., Weng, Y., Kerman, H., Freed, J. A., Cool, J. A., Kanjee, Z., Lane, K. P., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Hom, J., Chen, J. H., & Rodman, A. (2025). GPT-4 assistance for improvement of physician performance on patient care tasks: A randomized controlled trial. Nature Medicine, 31(4), 1233–1238. https://doi.org/10.1038/s41591-024-03456-y

Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Rodman, A., & Chen, J. H. (2024). Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial. JAMA Network Open, 7(10), e2440969. https://doi.org/10.1001/jamanetworkopen.2024.40969

Greenblatt, R., Cotra, A., & Wijk, H. (2026, August 26). Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18(7), 1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527

Hugging Face. (2026, July 16). Security incident disclosure: July 2026. https://huggingface.co/blog/security-incident-july-2026

Jones, C. R., & Bergen, B. K. (2025). Large Language Models Pass the Turing Test (Version 1). arXiv. https://doi.org/10.48550/arxiv.2503.23674

Lighthill, J. (1973). Artificial intelligence: A general survey [Artificial Intelligence: A Paper Symposium]. Science Research Council.

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/bf02478259

Meng, X., Yan, X., Zhang, K., Liu, D., Cui, X., Yang, Y., Zhang, M., Cao, C., Wang, J., Wang, X., Gao, J., Wang, Y.-G.-S., Ji, J.-M., Qiu, Z., Li, M., Qian, C., Guo, T., Ma, S., Wang, Z., … Tang, Y.-D. (2024). The application of large language models in medicine: A scoping review. iScience, 27(5), 109713. https://doi.org/10.1016/j.isci.2024.109713

Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2017). Deep learning for healthcare: Review, opportunities and challenges. Briefings in Bioinformatics, 19(6), 1236–1246. https://doi.org/10.1093/bib/bbx044

OpenAI. (2026, August 26). The hugging face incident and the road ahead. https://openai.com/index/hugging-face-incident-and-the-road-ahead/

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408. https://doi.org/10.1037/h0042519

Schier, R. (2018). Artificial Intelligence and the Practice of Radiology: An Alternative View. Journal of the American College of Radiology, 15(7), 1004–1007. https://doi.org/10.1016/j.jacr.2018.03.046

Schmidt, C. (2017). M. D. Anderson Breaks With IBM Watson, Raising Questions About Artificial Intelligence in Oncology. JNCI: Journal of the National Cancer Institute, 109(5). https://doi.org/10.1093/jnci/djx113

Special Competitive Studies Project, & Gallup. (2026, September 16). Looking to the future: American priorities for AI in 2026. https://www.prnewswire.com/news-releases/new-scsp-gallup-report-americans-agree-on-how-the-country-should-prepare-for-ai-302880594.html

The Guardian. (2026, September 28). AI godfathers warn of runaway “intelligence explosion”. https://www.theguardian.com/technology/2026/sep/28/ai-godfathers-warn-of-runaway-intelligence-explosion

Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, LIX(236), 433–460. https://doi.org/10.1093/mind/lix.236.433

WIRED. (2026). Timnit gebru believes there is no “existential threat” from AI. https://www.wired.com/story/the-big-interview-podcast-timnit-gebru/

Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., Ghous, M., & Singh, K. (2021). External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Internal Medicine, 181(8), 1065. https://doi.org/10.1001/jamainternmed.2021.2626

Yu, V. L., Fagan, L. M., Wraith, S. M., Clancey, W. J., Scott, A. C., Hannigan, J., Blum, R. L., Buchanan, B. G., & Cohen, S. N. (1979). Antimicrobial selection by a computer. A blinded evaluation by infectious diseases experts. JAMA, 242(12), 1279–1282. https://www.ncbi.nlm.nih.gov/pubmed/480542