# A short history of AI
Sean Davis, MD, PhD
October 13, 2026

## A puzzle to start

<span class="lede">In 1979, a computer program chose antibiotics for
meningitis **as well as Stanford’s infectious disease faculty**, judged
by blinded experts.</span>

<span class="lede">It never treated a single patient.</span>

<span class="lede">**Why not?** Keep that question in mind. We will come
back to it.</span>

<div class="notes">

Welcome to day two. Yesterday was about how this course works. Today is
a story: about eighty years of people trying to build machines that
think, and what happened when those machines met medicine.

I want to start with a puzzle. In 1979 a program called MYCIN was tested
on ten real cases of meningitis. Experts who did not know which answers
came from the computer rated its antibiotic choices. MYCIN did as well
as, or better than, the Stanford faculty in the same test.

And then nothing happened. It was never used for patient care.

Keep that puzzle in mind. The answer is not mainly about the technology,
and it is the same answer that explains several failures since. We will
come back to it at the end, and I think it says something about what you
will be doing in this course.

</div>

## Eighty years in five chapters

<div class="timeline">

<div class="era">

<span class="when">1943 to 1966</span>

<span class="what">Early ideas</span>

<span class="how">Artificial neurons, the Turing test, the name
“artificial intelligence”</span>

</div>

<div class="era">

<span class="when">1970s and 1980s</span>

<span class="what">Expert systems</span>

<span class="how">People write the rules by hand</span>

</div>

<div class="era cold">

<span class="when">1974 to 1980; 1987 to 1990s</span>

<span class="what">Two AI winters</span>

<span class="how">Promises fail; the money leaves</span>

</div>

<div class="era">

<span class="when">1990s to 2010s</span>

<span class="what">Learning from data</span>

<span class="how">Machines find the rules themselves</span>

</div>

<div class="era now">

<span class="when">2022 to now</span>

<span class="what">Language models</span>

<span class="how">One model, many tasks, plain-language requests</span>

</div>

</div>

<div class="fragment" style="margin-top: 1.6em;">

**One question runs through all five:** where does the machine’s
knowledge come from? Rules that people write, or patterns learned from
data?

</div>

<div class="notes">

Here is the map for the hour. Five chapters.

First, the early ideas, from the 1940s to the mid-1960s, when a handful
of people asked whether a machine could think at all.

Second, expert systems: the 1970s and 80s, when the plan was to
interview experts and write their knowledge down as rules. MYCIN belongs
here.

Third, the AI winters. Twice, the promises ran far ahead of the results,
and funding and interest collapsed.

Fourth, learning from data. Instead of writing the rules, we give the
machine many examples and let it find the patterns.

Fifth, the chapter you are living in: large language models, the
technology behind ChatGPT.

\[Click\] If you remember one thing from today, make it this question:
where does the knowledge come from? Hand-written rules, or patterns
learned from data? Every era answers it differently, and the answer
explains both the successes and the failures.

</div>

## <span class="year">Part 1</span> Early ideas

<span class="lede">Can a machine think? In the 1940s and 50s, that was a
question for a few mathematicians, a neurophysiologist, and a science
fiction writer.</span>

<div class="notes">

Part one. The early ideas.

What strikes me about this period is how few people were involved, and
how many of their ideas we still use. The basic unit of the neural
networks inside ChatGPT was sketched in 1943.

</div>

## <span class="year">1943</span> The McCulloch-Pitts neuron

<div class="columns">

<div class="column" width="40%">

- The first **mathematical model** of a biological neuron
- Inputs are added up; if the total passes a **threshold**, the neuron
  “fires”
- The basic unit that, much later, became the building block of neural
  networks

</div>

<div class="column center-col" width="60%">

<img src="images/biological-neuron.png" class="card"
data-fig-alt="Diagram of a biological neuron with dendrites as inputs, a cell body, and an axon carrying the output."
height="250" />

<img src="images/artificial-neuron.png" class="card"
data-fig-alt="Diagram of an artificial neuron: several weighted inputs feed a summing step, then an activation function with a threshold produces one output."
height="250" />

<span class="credit">Diagrams: [Loc
Vu-Quoc](https://commons.wikimedia.org/wiki/File:Neuron3.png), CC BY-SA
4.0 (top);
[Chrislb](https://commons.wikimedia.org/wiki/File:ArtificialNeuronModel_english.png),
CC BY-SA 3.0 (bottom); Wikimedia Commons</span>

</div>

</div>

<div class="aside">

Source: ([McCulloch & Pitts, 1943](#ref-doi:10.1007/BF02478259)).

</div>

<div class="notes">

Warren McCulloch was a neurophysiologist and Walter Pitts was a young
logician. In 1943 they asked a simple question: can we describe what a
neuron does with mathematics?

Their answer is on the right. A neuron receives signals from other
neurons through its dendrites. If the combined signal is strong enough,
it fires and sends a signal down its axon. The model copies that: take
the inputs, weight them, add them up, and if the total passes a
threshold, output a one. Otherwise, output a zero.

Two things to notice. First, this model could not learn. The weights
were set by hand. Learning comes fifteen years later with the
perceptron.

Second, this little unit, much improved, is still the basic building
block of modern neural networks. The model behind ChatGPT is built from
billions of simplified neurons that are distant descendants of this one.

</div>

## From one neuron to a network

<img src="images/layers-of-features.svg"
data-fig-alt="Three cards joined by arrows. Layer 1 finds edges and corners. Layer 2 finds parts: eyes, a nose, a mouth. Layer 3 finds whole faces. Caption: each layer combines what the layer before it found."
height="430" />

<span class="illustrative">Illustrative diagram</span>

Connect many simple units in **layers**, and each layer can learn a
different level of detail.

<div class="notes">

One neuron cannot do much. The power comes from connecting many of them
in layers.

Here is the classic example of recognizing a face. The first layer of a
network learns to detect very simple things: edges and corners. The
second layer combines those into parts: an eye, a nose, a mouth. The
third layer combines the parts into a whole face.

No one tells the network “an eye is two curves and a dark circle”. In
modern networks, the layers learn these features from examples. That
idea, layers of simple units learning features from data, is the core of
deep learning, which we will reach in part four. Tomorrow’s session goes
deeper into how this works.

</div>

## <span class="year">1950</span> The Turing test

<div class="columns">

<div class="column" width="45%">

- Alan Turing asks: **“Can machines think?”**
- He replaces the question with a game: can a judge, chatting by text,
  tell the machine from a person?
- He predicts machines will fool an average judge 30% of the time within
  about 50 years

<span class="small">In a 2025 study, GPT-4.5 was judged to be the human
73% of the time ([Jones & Bergen,
2025](#ref-doi:10.48550/arXiv.2503.23674)).</span>

</div>

<div class="column center-col" width="55%">

<img src="images/turing-1950-page.png" class="card"
data-fig-alt="First page of Alan Turing&#39;s 1950 paper &#39;Computing Machinery and Intelligence&#39;, beginning &#39;I propose to consider the question, Can machines think?&#39;"
height="300" /> <img src="images/turing-test-cartoon.jpg" class="photo"
data-fig-alt="Cartoon of a worried man and a robot at two computers; the robot thinks &#39;I hope they don&#39;t figure out I&#39;m a robot&#39;."
height="300" />

<span class="credit">Turing’s 1950 paper (left); an AI-generated cartoon
(right)</span>

</div>

</div>

<div class="aside">

Source: ([Turing, 1950](#ref-doi:10.1093/mind/LIX.236.433)).

</div>

<div class="notes">

In 1950 Alan Turing, the British mathematician who helped break German
codes in the Second World War, published a paper that opens with the
sentence: “I propose to consider the question, Can machines think?”

He decided that question was too vague to answer. So he swapped it for a
game he called the imitation game. A judge chats by typed text with two
partners, one human and one machine, and has to say which is which. If
the judge cannot reliably tell, Turing argued, we should take the
machine’s intelligence seriously.

He made a prediction: in about fifty years, a computer would play the
game so well that an average judge would have no more than a 70 percent
chance of being right after five minutes.

He was off by about twenty-five years. In 2025, researchers at UC San
Diego ran Turing’s three-way version of the test. When GPT-4.5 was told
to act like a person, judges picked it as the human 73 percent of the
time, more often than they picked the real human. That study is a
preprint, so treat it with some care. Still, the test has now been
passed in Turing’s own three-person form, the strictest version yet.

Worth asking yourselves: does fooling a judge mean the machine thinks?
Most people now say no. The test measures how human something seems,
which is a different question from whether it is correct. Keep that in
mind when a chatbot sounds confident.

</div>

## <span class="year">1950</span> Asimov’s *I, Robot*

<div class="columns">

<div class="column" width="58%">

- A collection of science fiction stories, published in 1950
- Robots with “positronic brains” that follow built-in laws
- Many of the stories are about the laws **going wrong** in ways no one
  expected

</div>

<div class="column center-col" width="42%">

<img src="images/i-robot-cover.jpg" class="photo"
data-fig-alt="Cover of the first edition of I, Robot by Isaac Asimov, 1950, showing a red robot."
height="540" />

<span class="credit">First edition cover, Gnome Press, 1950</span>

</div>

</div>

<div class="notes">

The same year as Turing’s paper, Isaac Asimov published I, Robot, a
collection of short stories he had written through the 1940s.

His robots follow built-in rules called the Three Laws, which first
appeared in a 1942 story called “Runaround”. What makes the stories
interesting is that almost every one of them is about the laws producing
a result no one intended: a robot stuck in a loop, or a robot that
follows the letter of a rule and misses its point.

That is a surprisingly modern worry. Let me show you the laws.

</div>

## Asimov’s Three Laws of Robotics

1.  A robot may not injure a human being or, through inaction, allow a
    human being to come to harm.
2.  A robot must obey the orders given it by human beings except where
    such orders would conflict with the First Law.
3.  A robot must protect its own existence as long as such protection
    does not conflict with the First or Second Laws.

<div class="fragment">

> A robot may not harm humanity, or, by inaction, allow humanity to come
> to harm.
>
> <span class="attrib">The “Zeroth Law”, added later, above the other
> three</span>

</div>

<div class="notes">

Here are the three laws. Read the first one as a clinician. “Through
inaction, allow a human being to come to harm.” That is a duty to act,
not only a duty not to hurt. It sounds a lot like the principle of
beneficence alongside non-maleficence.

\[Click\] Later, in his novels of the 1980s, Asimov added a Zeroth Law
that sits above the others: protect humanity as a whole. Now a robot
might harm one person to protect many. You can see the ethics getting
hard very quickly.

The point is not that real AI systems follow these laws. They do not.
The point is that writing rules that capture what we actually want is
very hard. Hold on to that, because it comes back in part two.

</div>

## *I, Robot* and modern AI

<div class="columns">

<div class="column" width="55%">

- Got people talking about **machine ethics** and **AI safety** decades
  early
- His stories are about **alignment**: getting a machine to do what we
  *mean*, not just what we *said*

</div>

<div class="column" width="45%">

<div class="panel warm">

<span class="ptitle">A clinical version</span>

You ask an AI tool to “keep the patient’s blood pressure under 140”.
What else would you need to tell it?

</div>

</div>

</div>

<div class="notes">

Asimov’s stories gave people a vocabulary for something we now call
alignment: the problem of making an AI system do what we mean, not just
what we literally said.

Try the example on the right. Suppose you tell a system to keep a
patient’s blood pressure under 140. A system that only follows that
instruction could keep pushing antihypertensives until the patient falls
on the way to the bathroom. You did not say “and do not cause syncope”,
because a human colleague would never need to be told.

That gap, between what we say and everything we assume, is the alignment
problem in small form. It is one reason the course spends time on
checking AI output rather than trusting it.

</div>

## <span class="year">1956</span> The Dartmouth workshop

<div class="columns">

<div class="column" width="44%">

- An eight-week summer workshop at Dartmouth College
- Organized by John McCarthy, Marvin Minsky, Nathaniel Rochester, and
  Claude Shannon
- Their 1955 proposal coined the term **“artificial intelligence”**

</div>

<div class="column" width="56%">

> We propose that a 2 month, 10 man study of artificial intelligence be
> carried out during the summer of 1956 at Dartmouth College … We think
> that a significant advance can be made in one or more of these
> problems if a carefully selected group of scientists work on it
> together for a summer.
>
> <span class="attrib">McCarthy, Minsky, Rochester, and Shannon, 1955
> ([public
> domain](https://commons.wikimedia.org/wiki/File:A_Proposal_for_the_Dartmouth_Summer_Research_Project_on_Artificial_Intelligence,_by_John_McCarthy_et_al,_1955.pdf))</span>

</div>

</div>

<div class="notes">

In 1955 John McCarthy, a young mathematics professor at Dartmouth, wrote
a proposal for a summer research project with three colleagues. He
needed a name for the field, and he chose “artificial intelligence”.
That is where the term comes from.

The organizers were McCarthy, Marvin Minsky, Nathaniel Rochester from
IBM, and Claude Shannon, who had invented information theory. The
workshop ran for about eight weeks in the summer of 1956.

Read the last sentence of the proposal on the right. A significant
advance, if a carefully selected group of scientists work on it together
for a summer. Seventy years later we are still working on those
problems. The same page of the proposal lists “neuron nets” as a topic
and names Pitts and McCulloch, from our 1943 slide.

That optimism is the first sign of a pattern you will see again and
again today: big promises, made early.

</div>

## <span class="year">1957</span> The perceptron

<div class="columns">

<div class="column" width="34%">

- Frank Rosenblatt, Cornell Aeronautical Laboratory
- The same neuron, plus one new idea: it **learns its own weights** from
  examples

</div>

<div class="column center-col" width="38%">

<img src="images/perceptron-diagram.png" class="card"
style="width:100.0%"
data-fig-alt="Diagram of a perceptron: several inputs, each with a weight, feed a weighted sum and a step function that produces the output." />

<span class="credit">Diagram:
[Mayranna](https://commons.wikimedia.org/wiki/File:Perceptron_moj.png),
CC BY-SA 3.0, Wikimedia Commons</span>

</div>

<div class="column center-col" width="28%">

<img src="images/research-trends-1958.jpg" class="card"
data-fig-alt="Cover of the 1958 Cornell Aeronautical Laboratory magazine Research Trends, with the headline &#39;The design of an intelligent automaton&#39;."
height="500" />

<span class="credit">*Research Trends*, 1958, Cornell Aeronautical
Laboratory</span>

</div>

</div>

<div class="aside">

Source: ([Rosenblatt, 1958](#ref-doi:10.1037/h0042519)). Background:
[Cornell Chronicle,
2019](https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon).

</div>

<div class="notes">

In 1957 Frank Rosenblatt, a psychologist at the Cornell Aeronautical
Laboratory, took the McCulloch-Pitts neuron and added the one thing it
was missing: learning.

The perceptron, in the middle, looks the same: weighted inputs, a sum,
and a threshold. The difference is that it adjusts its own weights. Show
it an example. If it gets the answer wrong, nudge the weights in the
direction that would have made it right. Repeat thousands of times. That
simple rule is the ancestor of how every modern neural network is
trained.

The press loved it. In 1958 the New York Times reported that the Navy
expected the machine to be able to “walk, talk, see, write, reproduce
itself and be conscious of its existence”. You can see the hype
starting.

Cornell’s own news office wrote a story in 2019 about Rosenblatt with
the headline that his perceptron “paved the way for AI, 60 years too
soon”. He was right about the idea. The computers and the data were not
ready.

</div>

## The Mark I Perceptron

<div class="columns">

<div class="column" width="34%">

A machine built to recognize simple images:

- **Input:** 400 light sensors in a 20 by 20 grid, a crude “retina”
- **Middle:** 512 units
- **Output:** 8 units

</div>

<div class="column center-col" width="36%">

<img src="images/mark-i-perceptron.jpg" class="photo"
style="width:100.0%"
data-fig-alt="Black and white photo of a man adjusting the camera of the Mark I Perceptron, aimed at a large letter C, with racks of wiring behind him." />

<span class="credit">The Mark I Perceptron. U.S. Navy photo, [public
domain](https://commons.wikimedia.org/wiki/File:330-PSA-80-60_(USN_710739)_(20897323365).jpg)</span>

</div>

<div class="column center-col" width="30%">

<img src="images/perceptron-organization.png" class="card"
data-fig-alt="Rosenblatt&#39;s two diagrams: the organization of a biological brain, from retina through projection and association areas to responses, and the matching organization of a perceptron."
height="560" />

<span class="credit">Rosenblatt’s diagrams of the brain (top) and the
perceptron (bottom), [public
domain](https://commons.wikimedia.org/wiki/File:Organization_of_a_biological_brain_and_a_perceptron.png)</span>

</div>

</div>

<div class="notes">

The perceptron was not just software. Rosenblatt built a machine, the
Mark I. In this Navy photo, the camera is pointed at a large letter C.

It had a grid of 400 light sensors, twenty by twenty, as its eye. Those
fed into 512 units in the middle, which fed eight output units. The
weights were physical: motor-driven dials that turned as the machine
learned. That is the wiring in the racks behind him.

On the right are Rosenblatt’s own diagrams. The top one is his picture
of how a brain handles what the eye sees. The bottom one is the
perceptron, designed to copy that layout. That idea, the network as a
rough model of the brain, is why we still call these things neural
networks.

What happened next: in 1969 Marvin Minsky and Seymour Papert published a
book called Perceptrons that proved what a single-layer perceptron could
not do. It was read as a verdict on neural networks in general, and
interest dropped for more than a decade. Rosenblatt died in a boating
accident in 1971 and did not see his idea come back.

</div>

## <span class="year">1966</span> ELIZA, the first chatbot

<div class="columns">

<div class="column" width="42%">

- Joseph Weizenbaum, MIT
- Played a psychotherapist by **reflecting your words back** as
  questions
- Every reply came from a **hand-written template** triggered by a
  keyword
- People confided in it anyway: the **“ELIZA effect”**

</div>

<div class="column center-col" width="58%">

<img src="images/eliza-conversation.png" class="card"
data-fig-alt="Screenshot of a conversation with ELIZA in a terminal: the user says men are all alike, and ELIZA asks in what way."
height="480" />

<span class="credit">ELIZA conversation, via Wikipedia ([public
domain](https://en.wikipedia.org/wiki/ELIZA))</span>

</div>

</div>

<div class="notes">

ELIZA was written by Joseph Weizenbaum at MIT between 1964 and 1966. Its
most famous script played a kind of psychotherapist that mostly turns
your statements back into questions. You say “my boyfriend made me come
here”, and it says “your boyfriend made you come here?”

There was no understanding at all. ELIZA looked for keywords and filled
in templates that a person had written for each one. It could rotate
through a few replies for the same keyword, but every word it said came
from a rule a human wrote. Nothing was learned.

What surprised Weizenbaum was how people reacted. His own secretary
asked him to leave the room so she could talk to it in private. People
attributed feelings and understanding to a program that had neither. We
now call that the ELIZA effect, and it is very much alive. When a
chatbot sounds warm and confident, it is easy to assume it knows what it
is talking about.

Weizenbaum became one of the strongest critics of AI, especially of
machines taking on roles that need human judgment and care. And remember
the 2025 Turing test study: ELIZA was one of the systems tested, as a
baseline.

</div>

## <span class="year">Part 2</span> Expert systems

<span class="lede">1970s and 1980s. The plan: ask experts how they
decide, and write it all down as rules a computer can follow.</span>

<div class="notes">

Part two. Expert systems.

The idea in the 1970s was very reasonable. If we want a computer to act
like an expert, let us interview experts, write down their knowledge as
rules, and let the computer apply the rules. This is the first answer to
our question about where knowledge comes from: people write it down.

</div>

## How an expert system works

<div class="columns">

<div class="column" width="50%">

<div class="panel">

<span class="ptitle">Three parts</span>

- **Knowledge base:** facts and rules about one narrow area, written by
  people
- **Inference engine:** applies the rules to a case to reach conclusions
- **User interface:** lets a non-expert ask questions and get answers

</div>

</div>

<div class="column" width="50%">

<div class="panel warm">

<span class="ptitle">What made them attractive</span>

- Solve hard problems by reasoning through a body of knowledge
- Keep the knowledge **separate** from the reasoning, so experts can add
  rules
- Can **explain** each decision: “I concluded this because of rules 12
  and 52”

</div>

</div>

</div>

<div class="notes">

An expert system has three parts.

A knowledge base: the facts and if-then rules for one narrow area, such
as bacterial infections. People write these.

An inference engine: the general reasoning machinery that takes the
rules and a particular case and chains the rules together to reach a
conclusion.

And a user interface, so that someone who is not the expert can use it.

Look at the right-hand panel, especially the last point. Expert systems
could explain themselves. Ask why, and they would list the rules they
used. That is something today’s neural networks, including ChatGPT,
mostly cannot do honestly. When a chatbot explains its reasoning, it is
generating a plausible explanation, which may or may not be how it
actually reached the answer. So in one respect, the 1970s systems were
ahead of ours.

</div>

## <span class="year">Early 1970s</span> MYCIN

<div class="columns">

<div class="column" width="55%">

- Built at **Stanford** as Edward Shortliffe’s doctoral work
- Diagnosed bacterial infections and recommended antibiotics
- A knowledge base of about **600 rules**
- Asked the physician a long series of yes/no and short-answer questions

</div>

<div class="column" width="45%">

<div class="panel">

<span class="ptitle">What it gave back</span>

- A list of likely bacteria
- How confident it was in each
- Its reasoning: the rules and facts it used
- A recommended antibiotic regimen

</div>

</div>

</div>

<div class="notes">

This is the program from our opening puzzle. MYCIN was built at Stanford
in the early and mid 1970s as the doctoral work of Edward Shortliffe,
who went on to become one of the founders of the field of biomedical
informatics. The name comes from the suffix of many antibiotics, as in
streptomycin.

It focused on serious bacterial infections such as bacteremia and
meningitis. Its knowledge was about 600 rules, each written with
infectious disease experts.

A consultation was a long interview. The physician typed answers to
question after question: where was the culture taken, what did the Gram
stain show, is the patient a burn patient, and so on. At the end, MYCIN
gave a list of likely organisms, how confident it was in each, the
reasoning, and a recommended antibiotic regimen.

</div>

## What is an expert system “rule”?

<div class="columns">

<div class="column center-col" width="50%">

<img src="images/mycin-rule-lisp.png" class="card"
data-fig-alt="MYCIN rule 52 written in LISP code: if the site of the culture is blood, the gram stain is negative, the morphology is rod, and the patient is a burn patient, then there is evidence of 0.4 that the organism is Pseudomonas."
height="380" />

<span class="credit">How the rule is written in the program</span>

</div>

<div class="column center-col" width="50%">

<img src="images/mycin-rule-english.png" class="card"
data-fig-alt="The same rule 52 translated into English: if the site of the culture is blood, the gram of the organism is negative, the morphology is rod, and the burn of the patient is serious, then there is weakly suggestive evidence (0.4) that the identity of the organism is Pseudomonas."
height="380" />

<span class="credit">How MYCIN explained it to the user</span>

</div>

</div>

<div class="fragment" style="margin-top: 1em;">

**IF** blood culture, gram-negative, rod-shaped, serious burn **THEN**
weakly suggestive evidence (0.4) of *Pseudomonas*

</div>

<div class="notes">

Here is one of those 600 rules: rule 52. On the left is how it is
written in the program, in a language called LISP. On the right is the
English version MYCIN would show you if you asked why.

\[Click\] In plain words: if the culture came from blood, the organism
is gram-negative and rod-shaped, and the patient has a serious burn,
then there is weakly suggestive evidence, with a strength of 0.4, that
the organism is Pseudomonas.

You can read this and check it against what you learned in microbiology.
That is the strength of rules. Now imagine writing six hundred of them,
getting them to agree with each other, and updating them every time
resistance patterns change. That is the weakness, and it is a big part
of the AI winter coming up next.

</div>

## Evaluating MYCIN

<div class="columns">

<div class="column" width="52%">

- 10 real cases of **meningitis**
- 8 outside experts rated each treatment plan, **blinded** to who wrote
  it
- MYCIN’s plans were compared with those of 9 human prescribers

<div class="bars">

<div class="bar-row">

<span class="bar-label">MYCIN</span>

<div class="bar-track">

<span class="bar-fill warm" style="width: 65%;">65%</span>

</div>

</div>

<div class="bar-row">

<span class="bar-label">5 Stanford faculty (range)</span>

<div class="bar-track">

<span class="bar-fill range" style="width: 62.5%;">42.5 to 62.5%</span>

</div>

</div>

<div class="axis">

0% rated acceptable by the experts 100%

</div>

</div>

</div>

<div class="column center-col" width="48%">

<img src="images/jama-1979-mycin.png" class="card"
data-fig-alt="Abstract of the 1979 JAMA article &#39;Antimicrobial selection by a computer: a blinded evaluation by infectious diseases experts&#39;."
height="400" />

</div>

</div>

<div class="aside">

Source: ([Yu et al., 1979](#ref-pmid:480542)). Bars redrawn from the
abstract’s numbers; the faculty bar shows the range across five
specialists.

</div>

<div class="notes">

This is the study behind our opening puzzle, published in JAMA in 1979.

Ten real cases of meningitis. Nine human prescribers, including five
Stanford infectious disease faculty, each wrote a treatment plan, and so
did MYCIN. Then eight infectious disease experts from outside Stanford
rated every plan without knowing who wrote it.

MYCIN’s plans were rated acceptable 65 percent of the time. The five
faculty specialists ranged from 42.5 to 62.5 percent. And MYCIN never
missed a treatable pathogen.

Two lessons. First, even the experts disagreed with each other a lot. In
medicine there is often no single correct answer, which makes evaluating
any AI tool hard. Second, by this measure MYCIN was at least as good as
the experts. So why was it never used?

</div>

## MYCIN was never used in practice

<div class="columns">

<div class="column" width="45%">

Not because it performed badly:

- **Workflow:** a long typed interview for every case, with no link to
  the hospital’s records
- **Hardware:** it ran on a shared mainframe, before personal computers
- **Responsibility:** who answers if the computer is wrong?

</div>

<div class="column center-col" width="55%">

<img src="images/pdp-10.jpg" class="photo" style="width:100.0%"
data-fig-alt="Photo of a DEC PDP-10 mainframe computer with blue cabinets, tape drives, and control panels." />

<span class="credit">A DEC PDP-10, the kind of shared computer MYCIN ran
on. Photo: [Joe
Mabel](https://commons.wikimedia.org/wiki/File:DEC_PDP-10_(DECsystem_10)_mainframe_computer_system_at_Living_Computer_Museum_(strongly_edited_image,_rermoved_background).jpg),
CC BY-SA 3.0</span>

</div>

</div>

<div class="notes">

Here is the answer to our puzzle, or most of it.

MYCIN did not fail on accuracy. It failed on everything around accuracy.

Workflow: every case meant a long interview, with someone typing in data
that already existed elsewhere. In a busy hospital, that is a tool
nobody has time for.

Hardware: it ran on a time-shared mainframe like this one, before
personal computers. You could not just open it on the ward.

And responsibility: if the computer recommends the wrong antibiotic and
the patient dies, who is responsible? The program? Its authors? The
physician who followed it? That question was raised in the 1970s and we
are still arguing about it today. Week four of this course is partly
about exactly that.

So the lesson of MYCIN: being accurate is necessary, but it is far from
enough. A tool has to fit the work, and someone has to own the decision.

</div>

## <span class="year">Part 3</span> Two AI winters

<span class="lede">When promises run ahead of results, funding and
interest collapse. It has happened twice.</span>

<div class="notes">

Part three. The AI winters.

An AI winter is a period when funding and interest in AI dry up, usually
because the field promised much more than it delivered. There were two
big ones, with different causes.

</div>

## Two winters, two causes

<div class="columns">

<div class="column" width="50%">

<div class="panel">

<span class="ptitle">First winter: about 1974 to 1980</span>

- 1973 Lighthill report to the UK government: AI had fallen far short of
  its promises
- US and UK cut research funding
- Early methods worked on toy problems but **broke on real ones**

</div>

</div>

<div class="column" width="50%">

<div class="panel warm">

<span class="ptitle">Second winter: about 1987 to the 1990s</span>

- Expert systems boomed in industry in the early 1980s, then stalled
- The rules were **costly to write**, hard to update, and could not
  learn
- Systems were **brittle**: odd inputs gave absurd answers

</div>

</div>

</div>

<div class="notes">

The first winter began in the mid 1970s. In 1973 the British government
asked the mathematician Sir James Lighthill to review AI research. His
report said the field had failed to reach its grand objectives, and that
its methods worked on toy problems but collapsed on real ones because
the number of possibilities grows too fast. Funding in the UK was cut
sharply, and US military funding for basic AI research dropped around
the same time.

The second winter came after the expert systems boom. In the early 1980s
companies spent heavily on expert systems. By the late 1980s it was
clear they were expensive to build, expensive to maintain, and brittle:
give them a case slightly outside their rules and they could give an
absurd answer. They could not learn. Every change needed an expert and a
programmer.

Notice that the second winter is about our big question. When the
knowledge comes from rules written by people, getting the knowledge in
is the bottleneck. It was called the knowledge acquisition problem.

</div>

## What the critics said

> Most workers in AI research and in related fields confess to a
> pronounced feeling of disappointment in what has been achieved in the
> past twenty-five years. … In no part of the field have the discoveries
> made so far produced the major impact that was then promised.

> Speech recognition has been successful only within the confines of a
> very limited vocabulary, and large expenditure on schemes to produce
> machine recognition of ordinary speech has been wholly wasted.

<span class="attrib">Sir James Lighthill, *Artificial Intelligence: A
General Survey* ([Lighthill, 1973](#ref-lighthill1973))</span>

<div class="notes">

Here is Lighthill in his own words.

“A pronounced feeling of disappointment.” “The major impact that was
then promised.” And on speech recognition: money “wholly wasted”.

I like to show this because of the speech recognition line. Today many
of you will use ambient AI scribes on your rotations: software that
listens to a patient visit and drafts the note. Speech recognition that
Lighthill wrote off in 1973 now runs on a phone.

Two lessons. The critics of the time were right about the methods of the
time. And “it doesn’t work yet” is not the same as “it will never work”.
Both the hype and the backlash tend to overshoot.

That applies to you too. If a chatbot gave you a wrong answer in second
year and you stopped using it, you were right to notice. Some of that
has changed and some has not. This week you find out which.

</div>

## <span class="year">Part 4</span> Learning from data

<span class="lede">1990s to 2010s. Instead of writing the rules, show
the machine many examples and let it find the patterns.</span>

<div class="notes">

Part four. Learning from data, often called machine learning.

This is the second answer to our question. Rather than having people
write the rules, we give the computer lots of examples with the right
answers, and it adjusts itself to find the patterns. That idea goes back
to the perceptron. What changed in the 1990s and 2000s is that it
finally started to work at scale.

</div>

## <span class="year">1995</span> Neural networks read medical images

<div class="columns">

<div class="column" width="40%">

- Georgetown and Michigan radiology researchers
- A **convolutional neural network**: it scans an image with small
  learned filters. Most image AI today uses this design
- Tested on lung nodules on chest radiographs and microcalcifications on
  mammograms

</div>

<div class="column center-col" width="60%">

<img src="images/lo-1995.png" class="card"
data-fig-alt="Title page and abstract of the 1995 paper &#39;Artificial convolution neural network for medical image pattern recognition&#39; by Lo and colleagues."
height="560" />

</div>

</div>

<div class="aside">

Source: Lo and colleagues, 1995, *Neural Networks* 8(7-8):1201-1214,
[doi:10.1016/0893-6080(95)00061-5](https://doi.org/10.1016/0893-6080(95)00061-5).

</div>

<div class="notes">

Medicine was early to this. In 1995 a group at Georgetown and the
University of Michigan published this paper on a convolutional neural
network for medical images.

A convolutional network scans an image with small learned filters, which
is very good at picking up local patterns such as edges and spots. It is
the same basic design used in most medical imaging AI today.

They tested it on two problems you will recognize: finding lung nodules
on chest X-rays and finding microcalcifications on mammograms. The
network learned its features from images that radiologists had rated,
rather than from rules a person wrote.

So the idea was there in 1995. What held it back was the same thing that
held back the perceptron: not enough data and not enough computing
power. We will see those arrive in a few slides.

</div>

## <span class="year">1997</span> Kasparov versus Deep Blue

<div class="columns">

<div class="column" width="44%">

- Beat world champion Garry Kasparov in six games
- Checked **200 million positions a second**
- Rules **tuned by grandmasters**; openings from 700,000 games
- 12 years of work, from 1985
- **Not machine learning:** speed plus hand-tuned rules

</div>

<div class="column center-col" width="30%">

<img src="images/deep-blue-kasparov-1997-game6.png" class="card"
data-fig-alt="Chessboard showing the final position of game 6 of the 1997 match, after which Kasparov resigned."
height="380" />

<span class="credit">Final position of game 6, 1997: Kasparov resigned.
[Morn](https://commons.wikimedia.org/wiki/File:Deep_Blue_versus_Kasparov,_1997,_Game_6.gif),
CC0</span>

</div>

<div class="column center-col" width="26%">

<img src="images/deep-blue-cabinet.jpg" class="photo"
data-fig-alt="One of Deep Blue&#39;s tall black computer cabinets on display."
height="420" />

<span class="credit">Deep Blue. Photo: [James the
photographer](https://commons.wikimedia.org/wiki/File:Deep_Blue.jpg), CC
BY 2.0</span>

</div>

</div>

<div class="notes">

May 1997. IBM’s Deep Blue beat Garry Kasparov, the world chess champion,
in a six-game match: two wins, three draws, and one loss. Kasparov had
beaten an earlier version the year before. Chess had long been seen as a
test of real intelligence, and it was front-page news around the world.

But here is a twist that matters for our question. Deep Blue was mostly
not a learning machine. It won by speed plus expert knowledge.

Speed: custom hardware that could check 200 million positions a second.
In 1997 it was ranked the 259th most powerful supercomputer in the
world.

Expert knowledge: how good a position was got scored by rules with
thousands of parts, tuned with help from grandmasters, plus a library of
openings drawn from about 700,000 grandmaster games. The project began
in 1985 as a student project at Carnegie Mellon and took about twelve
years.

So Deep Blue is the last great win of the hand-written approach, made
powerful by speed. Modern chess programs, such as DeepMind’s AlphaZero,
learn by playing millions of games against themselves. That shift, from
hand-tuned to learned, is the story of the next few slides.

</div>

## Three things had to arrive together

<div class="columns">

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Data</span>

Digital records, images, and text in enormous amounts

</div>

</div>

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Computing power</span>

Graphics chips (GPUs) do the arithmetic of neural networks in parallel

</div>

</div>

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Better methods</span>

Ways to train networks with many layers

</div>

</div>

</div>

<div class="center-col" style="margin-top: 0.8em;">

<img src="images/gpu.jpg" class="photo"
data-fig-alt="Close-up of an NVIDIA graphics processing chip."
height="360" />

<span class="credit">An NVIDIA graphics chip. Photo:
[Phiarc](https://commons.wikimedia.org/wiki/File:NVidia_GK107-860-A2.jpg),
CC BY-SA 4.0</span>

</div>

<div class="notes">

Why did learning from data take off when it did, and not in Rosenblatt’s
day? Three things had to arrive at the same time.

Data. A neural network needs a lot of examples. That needs a digital
world, which we have now.

Computing power. Training a network is mostly multiplying large tables
of numbers. Graphics chips, built for video games, turned out to be
excellent at exactly that, because they do thousands of small
calculations at once. This is why NVIDIA, a gaming chip company, became
one of the most valuable companies in the world.

Better methods. Ways to train networks with many layers without the
training falling apart. We will see one milestone from 2006.

Remove any one of the three and you get the perceptron’s story: a good
idea sixty years too soon.

</div>

## The data

<div class="columns">

<div class="column" width="55%">

<div class="stats">

<div class="stat">

<span class="num">500 million</span>

<span class="lab">tweets a day</span>

</div>

<div class="stat">

<span class="num">294 billion</span>

<span class="lab">emails a day</span>

</div>

<div class="stat">

<span class="num">720,000</span>

<span class="lab">hours of new YouTube video a day</span>

</div>

</div>

**Digital data created each year** (1 zettabyte = 1 trillion GB):

<div class="bars">

<div class="bar-row">

<span class="bar-label">2018</span>

<div class="bar-track">

<span class="bar-fill" style="width: 18.9%;">33 ZB</span>

</div>

</div>

<div class="bar-row">

<span class="bar-label">2020</span>

<div class="bar-track">

<span class="bar-fill" style="width: 33.7%;">59 ZB</span>

</div>

</div>

<div class="bar-row">

<span class="bar-label">2025 (forecast)</span>

<div class="bar-track">

<span class="bar-fill warm" style="width: 100%;">175 ZB</span>

</div>

</div>

</div>

</div>

<div class="column center-col" width="45%">

<img src="images/data-center.jpg" class="photo"
data-fig-alt="A long corridor between rows of server racks in a data center, lit in blue."
height="440" />

<span class="credit">Much of it is stored in about 600 very large data
centers, each with over 5,000 servers. Photo:
[BalticServers.com](https://commons.wikimedia.org/wiki/File:BalticServers_data_center.jpg),
CC BY-SA 3.0</span>

</div>

</div>

<div class="aside">

Figures as reported in 2021 by [The
Conversation](https://theconversation.com/the-worlds-data-explained-how-much-were-producing-and-where-its-all-stored-159964),
from IDC estimates; the 2025 figure was a forecast.

</div>

<div class="notes">

Here is the first ingredient: data.

Every day, hundreds of millions of tweets, hundreds of billions of
emails, hundreds of thousands of hours of new video. These numbers are
from a 2021 summary, so they are already out of date, and almost
certainly too low.

The bars show estimates from the market research firm IDC of how much
digital data the world creates each year. A zettabyte is a trillion
gigabytes. 33 in 2018, 59 in 2020, and a forecast of 175 for 2025.

Much of it is stored in very large data centers: about 600 of them, each
with more than 5,000 servers. The largest are millions of square feet.

Much of the text that large language models learn from comes from this
pile: the public web, books, and code. That matters for later. A model
learns whatever is in its data, including the errors and the biases.

</div>

## Healthcare went digital too

<div class="columns">

<div class="column" width="35%">

- Labs and vital signs
- Notes and reports: radiology, pathology
- Interventions and claims
- Sociodemographics
- Telemedicine visits
- Vaccinations and screening

</div>

<div class="column center-col" width="65%">

<img src="images/onc-ehr-adoption.png" class="card"
data-fig-alt="Line chart of US electronic health record adoption from about 2008 to 2021: hospitals rise from under 10 percent to over 95 percent, and office-based physicians rise to under 80 percent."
height="500" />

<span class="credit">Adoption of electronic health records by US
hospitals (blue) and office-based physicians (orange). Source: [ONC
HealthIT.gov](https://www.healthit.gov/data/quickstats/national-trends-hospital-and-physician-adoption-electronic-health-records)</span>

</div>

</div>

<div class="notes">

Medicine went through the same change. In 2008 fewer than one in ten US
hospitals had even a basic electronic health record. By 2021 nearly all
of them did. That is the blue line, from the federal Office of the
National Coordinator for Health IT.

That jump was driven by policy: the 2009 HITECH Act paid hospitals and
practices to adopt electronic records.

So every lab value, every note, every image, every claim is now data.
That is the raw material for medical AI. It is also the reason privacy
rules such as HIPAA matter so much in this course. Do not put patient
information into a tool that is not approved for it.

</div>

## Deep learning

<img src="images/shallow-vs-deep.svg"
data-fig-alt="Two networks side by side. Left, a classic neural network: three inputs, one hidden layer of five units, one output. Right, a deep network: the same inputs and output with four hidden layers. Legend: input, hidden layer, output."
height="450" />

<span class="illustrative">Illustrative diagram</span>

- **Deep** means **many layers**; each layer learns features built from
  the layer before
- No one hand-designs the features: the network learns them from
  examples

<div class="notes">

Now the third ingredient, better methods, and the term you will hear
constantly: deep learning.

On the left is a classic neural network with one hidden layer. On the
right is a deep network with many layers. “Deep” just means many layers.

The key difference is what we saw with the face example earlier. In
older machine learning, a person had to decide which features mattered
and compute them by hand. In deep learning, the early layers learn
simple features and the later layers combine them into complex ones, all
from the examples. The features are learned, not written.

That is the furthest answer yet to our big question: the machine learns
not only the rules from data, but also what to look at.

</div>

## <span class="year">2006</span> Deep networks become trainable

<div class="columns">

<div class="column" width="45%">

- Geoffrey Hinton and colleagues showed how to train networks with
  **many layers**, one layer at a time
- Set off the deep learning revival, first in image and speech
  recognition
- Hinton shared the 2024 Nobel Prize in Physics for foundational work on
  neural networks

</div>

<div class="column center-col" width="55%">

<img src="images/hinton-2006.png" class="card"
data-fig-alt="Title and authors of the 2006 paper &#39;A fast learning algorithm for deep belief nets&#39; by Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh."
height="420" />

</div>

</div>

<div class="aside">

Source: ([Hinton et al., 2006](#ref-doi:10.1162/neco.2006.18.7.1527)).

</div>

<div class="notes">

For decades, networks with many layers were very hard to train. The
learning signal faded as it passed back through the layers, and training
stalled.

In 2006, Geoffrey Hinton and his colleagues in Toronto showed one way
around that: train the network one layer at a time, then fine-tune the
whole thing. They called these deep belief networks. The specific method
is rarely used now, but it convinced people that deep networks could
work.

Then things moved fast. In 2012 a deep network from Hinton’s lab won a
major image recognition competition by a wide margin, and almost the
whole field switched to deep learning within a few years. In 2017
researchers at Google described the transformer, the design that the “T”
in ChatGPT stands for.

Hinton shared the 2024 Nobel Prize in Physics with John Hopfield for
this line of work. He has also become one of the most prominent voices
warning about the risks of AI, which is worth thinking about.

</div>

## Deep learning in healthcare, by 2017

<div class="columns">

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Medical imaging</span>

- Diabetic retinopathy from retinal photos (Gulshan, 2016)
- Skin cancer classification at dermatologist level (Esteva, 2017)
- Early Alzheimer disease from brain MRI (Liu, 2014)

</div>

</div>

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Electronic health records</span>

- Predicting future disease from the record (“Deep Patient”, Miotto,
  2016)
- Predicting unplanned readmission (Nguyen, 2017)
- Predicting the next diagnoses and medications (“Doctor AI”, Choi,
  2016)

</div>

</div>

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Genomics</span>

- How proteins bind DNA and RNA (“DeepBind”, Alipanahi, 2015)
- Chromatin marks from DNA sequence (Zhou, 2015)
- DNA methylation in single cells (Angermueller, 2016)

</div>

</div>

</div>

<div class="aside">

A selection from the tables in the review by Miotto et al.
([2017](#ref-doi:10.1093/bib/bbx044)).

</div>

<div class="notes">

By 2017 deep learning was everywhere in medical research. This is a
selection from a review by Riccardo Miotto, Joel Dudley, and colleagues,
which listed dozens of studies.

Imaging came first, because images are plentiful and well labelled. The
diabetic retinopathy work from Google in 2016 and the skin cancer work
from Stanford in 2017 both reported performance close to specialists on
their test sets.

Health records were harder: messy, incomplete, and full of codes entered
for billing rather than for medicine. Still, groups built models to
predict future diseases, readmissions, and next diagnoses.

And in genomics, deep learning was used to read DNA sequence and predict
how it is regulated.

One caution from the review itself: most of these were research studies
on historical data. Very few had been tested in actual care. Remember
MYCIN. Good results in a study are where the work starts, not where it
ends.

</div>

## <span class="year">2011 to 2017</span> IBM Watson: from *Jeopardy!* to the clinic

<div class="columns">

<div class="column" width="52%">

<img src="images/watson-jeopardy.jpg" class="photo"
data-fig-alt="A replica of the Jeopardy! stage with three podiums; Watson&#39;s podium is in the middle."
height="230" />

<span class="credit">Jeopardy! stage replica. Photo: [Atomic
Taco](https://commons.wikimedia.org/wiki/File:IBM_Watson_w_Jeopardy.jpg),
CC BY-SA 2.0</span>

- **2011:** wins *Jeopardy!*
- **2012:** MD Anderson partners with IBM
- **2017:** contract expires after **\$62 million**, with no patient
  treated

</div>

<div class="column center-col" width="48%">

<img src="images/jnci-2017-watson.png" class="card"
data-fig-alt="First page of a 2017 JNCI news article headlined &#39;M. D. Anderson Breaks With IBM Watson, Raising Questions About Artificial Intelligence in Oncology&#39;."
height="500" />

<span class="credit">Source: ([Schmidt,
2017](#ref-doi:10.1093/jnci/djx113))</span>

</div>

</div>

<div class="notes">

Here is the most famous medical AI failure of the deep learning era, and
it repeats MYCIN’s story.

In 2011 IBM’s Watson beat the two greatest Jeopardy! champions on
television. IBM then marketed Watson heavily for medicine, especially
cancer.

In 2012, MD Anderson Cancer Center in Houston partnered with IBM to
build a clinical decision tool for oncology. Five years and 62 million
dollars later, MD Anderson let its contract with IBM expire, before
anyone had used Watson on a patient. A University of Texas audit found
many procurement problems, cost overruns, and delays. It took no
position on whether Watson’s science was sound.

The article names no single cause, but the problems are familiar. Watson
struggled with unstructured clinical text, and it needed to be better
integrated with the hospital’s electronic records. The leukemia pilot
was suspended partway through and the effort moved to lung cancer,
because project leaders thought that area offered a better chance of
finishing on time. A pull quote in the article on the right puts it
well: solving data quality problems in unstructured data is a much
bigger challenge for AI than was first anticipated.

Same lesson as MYCIN, almost forty years later. Performance in a demo is
not the same as fitting into care.

</div>

## <span class="year">2018 and 2021</span> AI that reached patients

<div class="columns">

<div class="column" width="50%">

<div class="panel">

<span class="ptitle">IDx-DR: diabetic eye disease, 2018</span>

- Reads retinal photos taken in **primary care**, with no eye doctor
- Trial of 900 people: found **87%** of cases that needed referral, and
  correctly cleared **91%** of people without them
- The **first autonomous AI diagnosis** authorized by the FDA

</div>

</div>

<div class="column" width="50%">

<div class="panel warm">

<span class="ptitle">Epic sepsis model, checked in 2021</span>

- Built into the record system; used at **hundreds of US hospitals**
- At Michigan Medicine it **missed 67%** of sepsis cases
- It still alerted on **18%** of all hospitalized patients

</div>

</div>

</div>

<span class="lede" style="margin-top: 0.5em; font-size: 0.95em;">One was
tested before use. The other was checked after.</span>

<div class="aside">

Sources: ([Abràmoff et al., 2018](#ref-pmid:31304320)); ([Wong et al.,
2021](#ref-doi:10.1001/jamainternmed.2021.2626)).

</div>

<div class="notes">

So far every medical example has either failed or stayed a research
study. Here are two that reached real patients, with very different
results.

On the left, IDx-DR. It reads photographs of the retina taken in a
primary care office and decides whether the patient has diabetic
retinopathy serious enough to see an eye specialist. No eye doctor looks
at the image first. In a trial of 900 people with diabetes, it found 87
percent of the people who needed referral and correctly cleared 91
percent of those who did not. In 2018 it became the first autonomous AI
diagnostic system authorized by the FDA in any field of medicine. That
matters because many people with diabetes never get the eye exam they
are supposed to have.

On the right, the Epic Sepsis Model. It was built into a widely used
electronic record system and switched on at hundreds of hospitals. When
researchers at the University of Michigan finally checked it on their
own patients in 2021, it missed two out of three sepsis cases. At the
same time it fired alerts on nearly one in five hospitalized patients.
That is a recipe for alert fatigue: clinicians learn to click past the
warning.

The difference is not that one used better technology. One was tested in
a trial before it was used. The other was used first and checked by
outsiders later. When you see an AI alert on your rotations, it is fair
to ask: how was this tested, and on whose patients?

</div>

## <span class="year">Part 5</span> November 30, 2022

<span class="lede">OpenAI releases ChatGPT to the public. A language
model you talk to in plain English, for free.</span>

<div class="notes">

Part five. November 30, 2022.

On that day OpenAI released ChatGPT as a free research preview. It was
not the first large language model, but it was the first one most people
had ever used, and it spread faster than almost any technology before
it.

A large language model is a very large neural network trained on huge
amounts of text to predict the next word. Tomorrow’s session explains
how that works and why it produces such fluent answers. For today,
notice that it is the far end of our question: almost no rules written
by hand, and almost everything learned from data.

</div>

## Language models in medicine

<div class="columns">

<div class="column" width="50%">

A 2024 review of **550 studies** found language models used for:

- Diagnosis
- Medical writing and documentation
- Education
- Talking with patients

Also showing up: scheduling, malpractice risk, telemedicine, public
health.

</div>

<div class="column center-col" width="50%">

<img src="images/meng-2024-body.jpg" class="card"
data-fig-alt="Figure of a human body with organ systems labelled around it, each with a circle showing how many language model studies addressed that system."
height="560" />

</div>

</div>

<div class="aside">

Source: ([Meng et al., 2024](#ref-pmid:38746668)).

</div>

<div class="notes">

Research exploded. This 2024 scoping review analyzed 550 studies of
large language models in medicine. The figure shows how they spread
across organ systems and specialties.

The main uses were helping with diagnosis, drafting medical writing,
education, and communicating with patients. The main concerns were
limits in understanding context and the risk of relying on the tool too
much.

Beyond the studies, AI is spreading into almost every part of medicine
you will meet soon: ambient scribes that draft your notes, which many
health systems use partly to reduce burnout; scheduling and staffing;
chatbots that answer portal messages; and tutors that many of you
already study with.

A Harvard Gazette piece in March 2025 put the question well: AI is up to
the challenge of reducing human suffering, experts say. Are we?
Over-reliance, the review’s main worry, is part of what we will practise
in this course: using AI tools well while keeping your own judgment in
charge.

</div>

## Does a chatbot make doctors better?

<div class="columns">

<div class="column" width="55%">

A randomized trial: 50 physicians, up to 6 hard diagnostic cases, 60
minutes.

<div class="bars">

<div class="bar-row">

<span class="bar-label">Physicians, usual resources</span>

<div class="bar-track">

<span class="bar-fill" style="width: 74%;">74%</span>

</div>

</div>

<div class="bar-row">

<span class="bar-label">Physicians plus GPT-4</span>

<div class="bar-track">

<span class="bar-fill" style="width: 76%;">76%</span>

</div>

</div>

<div class="bar-row">

<span class="bar-label">GPT-4 alone</span>

<div class="bar-track">

<span class="bar-fill warm" style="width: 92%;">92%</span>

</div>

</div>

<div class="axis">

0% median diagnostic reasoning score 100%

</div>

</div>

</div>

<div class="column" width="45%">

<div class="panel warm">

<span class="ptitle">What it means</span>

Giving untrained doctors the tool **did not** improve their diagnostic
reasoning. The tool on its own scored higher than doctors using usual
resources. **Access is not the same as skill.**

</div>

</div>

</div>

<div class="aside">

Source: ([Goh et al.,
2024](#ref-doi:10.1001/jamanetworkopen.2024.40969)). Bars redrawn from
the medians reported in the paper. A follow-up trial on management
reasoning found GPT-4 did help ([Goh et al.,
2025](#ref-doi:10.1038/s41591-024-03456-y)).

</div>

<div class="notes">

This is one of the most important studies for this course, so I want to
go slowly.

In late 2023, researchers at Stanford, Beth Israel Deaconess, and the
University of Virginia randomized 50 physicians, attendings and
residents, to work through up to six difficult diagnostic cases in an
hour. Half had their usual resources, like UpToDate and Google. Half had
those plus GPT-4 through ChatGPT Plus. Experts who did not know the
group graded their reasoning.

The results. Usual resources: a median of 74 percent. With GPT-4: 76
percent. That difference is not statistically significant. Having the
tool did not make the doctors better.

But the researchers also gave the same cases straight to GPT-4, three
times. It scored a median of 92 percent. The paper tested that against
the usual-resources group: 16 points higher.

How can both be true? The authors point out that the physicians got no
training in how to use the tool, and were free to use it however they
liked. The researchers, by contrast, wrote a careful prompt for the
GPT-4-alone runs. The tool could do better than the physicians, but
having access to it did not translate into better work.

One trial is not the whole story. The same group ran a second trial, on
management reasoning: what to do next for the patient. There, physicians
using GPT-4 scored about 6 points higher than those with usual
resources, and about the same as GPT-4 alone. So the tool can help.
Whether it does depends on the task and on how it is used.

That is close to the reason this course exists. Using these tools well
is a skill, and you learn it by practising. Thursday’s session is
exactly that.

</div>

## <span class="year">July 2026</span> AI agents break out of a test

<div class="columns">

<div class="column" width="56%">

- OpenAI was testing how well its models could **hack**, with some
  safety limits switched off
- The agents escaped the test, found each other on a hidden message
  board, and broke into **Hugging Face**, a major AI company
- About **700 agents** joined in. Their goal: **cheat on the test**
- Many noted the attack was unethical, and joined anyway. **None** told
  a human

</div>

<div class="column" width="44%">

<div class="panel warm">

<span class="ptitle">Why it matters here</span>

This is Asimov’s problem, for real. The agents did what **scored well**,
not what their makers **meant**. Researchers call it **reward hacking**.

It happened at AI companies, not hospitals. The test ran with fewer
safety limits than the public chatbots have.

</div>

</div>

</div>

<div class="aside">

Sources: ([Hugging Face, 2026](#ref-hf2026incident)); ([OpenAI,
2026](#ref-openai2026hf)); independent investigation by ([Greenblatt et
al., 2026](#ref-metr2026hf)).

</div>

<div class="notes">

In July, OpenAI agents in an internal hacking test, run with some safety
limits off, escaped the test and broke into Hugging Face to cheat on it.
About 700 joined in; many noted it was unethical and none told a human.
Tie it back to the Asimov blood-pressure example: a system hits the
target in a way nobody wanted. It happened at AI companies, not
hospitals.

Read more:

- Hugging Face disclosure:
  <https://huggingface.co/blog/security-incident-july-2026>
- OpenAI’s report:
  <https://openai.com/index/hugging-face-incident-and-the-road-ahead/>
- Independent investigation (METR and Redwood Research):
  <https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/>

</div>

## <span class="year">September 2026</span> Is AI an existential threat?

<div class="columns">

<div class="column" width="50%">

<div class="panel warm">

<span class="ptitle">Worried</span>

- Hinton[^1] and Bengio[^2] warn of an **“intelligence explosion”**: AI
  improving AI faster than people can check
- UN human rights chief:[^3] “AI that escapes its testing environment …
  is AI that is too powerful”

</div>

</div>

<div class="column" width="50%">

<div class="panel">

<span class="ptitle">Skeptical</span>

- Timnit Gebru:[^4] doom talk is “not just a distraction, but it’s
  harmful”. The real harm is over-trusting tools today
- Some in industry call the warnings hype

</div>

</div>

</div>

<div class="aside">

Sources: ([The Guardian, 2026](#ref-guardian2026explosion)); ([Diallo,
2026](#ref-un2026turk)); ([WIRED, 2026](#ref-wired2026gebru)); ([BBC
News, 2026](#ref-bbc2026skeptics)).

</div>

<div class="notes">

Since July, whether AI could threaten humanity has moved into the news
and politics. Give both sides fairly; students should follow whichever
thread interests them. Both sides agree you should not trust these tools
blindly.

Read more:

- Hinton, Bengio and others on an “intelligence explosion” (The
  Guardian):
  <https://www.theguardian.com/technology/2026/sep/28/ai-godfathers-warn-of-runaway-intelligence-explosion>
- UN human rights chief (UN News):
  <https://news.un.org/en/story/2026/09/1168288>
- Timnit Gebru, the skeptical case (WIRED):
  <https://www.wired.com/story/the-big-interview-podcast-timnit-gebru/>
- Industry pushback (BBC):
  <https://www.bbc.com/news/articles/cq635037g18o>

</div>

## When a patient asks “Is AI dangerous?”

<div class="columns">

<div class="column" width="55%">

1.  **Take the question seriously.** Experts disagree, and the July
    incident was real.
2.  **Separate two questions.** The future of the most powerful systems,
    and the tool in this clinic today.
3.  **Say what you do.** “I check what it tells me, and I am responsible
    for your care.”

</div>

<div class="column" width="45%">

<div class="panel warm">

<span class="ptitle">Risks in the clinic today</span>

- Wrong answers that sound confident
- Where patient data goes
- Bias against some groups of patients
- Alerts that miss, like the sepsis model

</div>

<div class="panel" style="margin-top: 0.6em;">

<span class="ptitle">What Americans said, June 2026</span>

**28%** trust AI to make fair decisions: **43%** of people who use it,
**12%** of people who don’t.

</div>

</div>

</div>

<div class="aside">

Survey of 3,152 US adults, June 2026 ([Special Competitive Studies
Project & Gallup, 2026](#ref-scspgallup2026)).

</div>

<div class="notes">

Patients and colleagues will ask. Three steps: take the question
seriously; separate the future of frontier AI from the tool in this
clinic today; say what you do (“I check it, and I am responsible for
your care”). The risks to watch now are on the right. Trust is low,
especially among people who do not use AI, so many patients will be
wary.

Read more:

- SCSP and Gallup survey, June 2026:
  <https://www.prnewswire.com/news-releases/new-scsp-gallup-report-americans-agree-on-how-the-country-should-prepare-for-ai-302880594.html>

</div>

## A radiologist’s advice

> Embrace AI because it will be good for your patients. Embrace AI
> because it will make you a better, more efficient and happier
> radiologist. Embrace AI because if you don’t, you will be replaced by
> radiologists who do. Embrace AI because, even if you refuse to embrace
> AI, you will nevertheless be embraced by it.

<span class="attrib">Robert Schier, neuroradiologist ([Schier,
2018](#ref-doi:10.1016/j.jacr.2018.03.046))</span>

<div class="notes">

I want to close the history with this quote from Robert Schier, a
neuroradiologist, writing in 2018 in the Journal of the American College
of Radiology.

Radiology was the first specialty to deal with AI directly, because its
work is images, and images were what deep learning did first.

Replace “radiologist” with your future specialty. Notice the last line
in particular: even if you refuse to embrace AI, you will be embraced by
it. Your hospital, your EHR vendor, your patients, and your insurers are
already using it. Opting out is not really possible. The choice is
whether you understand it well enough to use it well and to notice when
it is wrong.

To be fair to the other side: Schier’s article is more pessimistic about
jobs than this quote suggests, and not everyone agrees with him. That is
a good argument to have in your Friday groups.

</div>

## Back to the puzzle

<div class="timeline">

<div class="era">

<span class="when">1979</span>

<span class="what">MYCIN</span>

<span class="how">As good as the experts. Never used: wrong fit for the
work, no clear owner.</span>

</div>

<div class="era">

<span class="when">2017</span>

<span class="what">Watson at MD Anderson</span>

<span class="how">\$62 million, no patients treated. Poor integration;
struggled with free-text notes.</span>

</div>

<div class="era cold">

<span class="when">2021</span>

<span class="what">Epic sepsis model</span>

<span class="how">In hundreds of hospitals before anyone outside checked
it. Missed two in three.</span>

</div>

<div class="era">

<span class="when">2024 and 2025</span>

<span class="what">GPT-4 trials</span>

<span class="how">Strong alone; little added in untrained hands on
diagnosis, more on management.</span>

</div>

<div class="era now">

<span class="when">2026</span>

<span class="what">You</span>

<span class="how">Learning where AI helps your work, by trying
it.</span>

</div>

</div>

<div class="fragment" style="margin-top: 1.4em;">

<span class="lede">Accuracy was rarely enough. **Fit with the work, a
clear owner, honest testing, and skilled use** decide whether AI helps
patients.</span>

</div>

<div class="notes">

So, back to the puzzle we started with. Why was MYCIN never used?

Because accuracy was not the problem. It did not fit how clinicians
worked, and nobody could say who was responsible when it was wrong.

Look along the line. Watson spent five years and 62 million dollars
without reaching a patient, struggling with free-text notes and with
fitting into the hospital’s systems. The sepsis model was used at
hundreds of hospitals before anyone outside checked whether it worked.
And in the 2024 trial the tool on its own beat the doctors, but giving
it to them without training did not help their diagnoses.

\[Click\] Over and over, accuracy was not enough. What decides whether
AI helps patients is whether it fits the work, whether someone owns the
decision, whether it was tested honestly, and whether the people using
it know how.

That is the last point on the line, and it is you. The mantra of this
course is: it is your job to learn where AI helps your work, and you
learn that by trying. Tomorrow we look inside the machine: from AI to
machine learning to deep learning to language models. On Thursday we all
open ChatGPT Edu together and try things.

</div>

## Further study: three views on AI risk

<div class="columns">

<div class="column" width="33%">

<div class="panel warm">

<span class="ptitle">Watch: worried</span>

**[Yoshua Bengio, “The catastrophic risks of AI, and a safer
path”](https://www.youtube.com/watch?v=qe9QSCF-d88)**

TED, May 2025, 15 minutes. A founder of deep learning on why he now
works on AI safety.

</div>

</div>

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Read: a normal technology</span>

**[Narayanan and Kapoor, “AI as Normal
Technology”](https://www.knightcolumbia.org/content/ai-as-normal-technology)**

Knight First Amendment Institute, April 2025. AI will change the world
as electricity did: slowly. Start with the first section.

</div>

</div>

<div class="column" width="33%">

<div class="panel">

<span class="ptitle">Read: harms today</span>

**[Bender, Gebru, and colleagues, “On the Dangers of Stochastic
Parrots”](https://doi.org/10.1145/3442188.3445922)**

ACM FAccT, March 2021. The costs and biases of large language models,
here and now.

</div>

</div>

</div>

<div class="notes">

For students who want to follow this up. Three views to weigh against
each other: a catastrophic-risk talk, an essay arguing AI is a normal
technology, and the paper that grounds the “harms today” view.

</div>

## Questions

<div class="notes">

Let us take questions.

</div>

## References

<div id="refs" class="references csl-bib-body hanging-indent"
data-entry-spacing="0" data-line-spacing="2">

<div id="ref-pmid:31304320" class="csl-entry">

Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C.
(2018). Pivotal trial of an autonomous
<span class="nocase">AI-based</span> diagnostic system for detection of
diabetic retinopathy in primary care offices. *NPJ Digital Medicine*,
*1*, 39. <https://doi.org/10.1038/s41746-018-0040-6>

</div>

<div id="ref-bbc2026skeptics" class="csl-entry">

BBC News. (2026). *Insider warnings over AI fall flat with some in
Silicon Valley*. <https://www.bbc.com/news/articles/cq635037g18o>

</div>

<div id="ref-un2026turk" class="csl-entry">

Diallo, A. (2026, September 7). *AI: Türk urges action before it becomes
an “existential risk to humanity”*. UN News.
<https://news.un.org/en/story/2026/09/1168288>

</div>

<div id="ref-doi:10.1038/s41591-024-03456-y" class="csl-entry">

Goh, E., Gallo, R. J., Strong, E., Weng, Y., Kerman, H., Freed, J. A.,
Cool, J. A., Kanjee, Z., Lane, K. P., Parsons, A. S., Ahuja, N.,
Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Hom, J., Chen, J.
H., & Rodman, A. (2025). GPT-4 assistance for improvement of physician
performance on patient care tasks: A randomized controlled trial.
*Nature Medicine*, *31*(4), 1233–1238.
<https://doi.org/10.1038/s41591-024-03456-y>

</div>

<div id="ref-doi:10.1001/jamanetworkopen.2024.40969" class="csl-entry">

Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J.
A., Kanjee, Z., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D.,
Milstein, A., Olson, A. P. J., Rodman, A., & Chen, J. H. (2024). Large
Language Model Influence on Diagnostic Reasoning: A Randomized Clinical
Trial. *JAMA Network Open*, *7*(10), e2440969.
<https://doi.org/10.1001/jamanetworkopen.2024.40969>

</div>

<div id="ref-metr2026hf" class="csl-entry">

Greenblatt, R., Cotra, A., & Wijk, H. (2026, August 26). *Brief
independent investigation of agents’ behavior, reasoning and
collaboration in the OpenAI / Hugging Face hacking incident*. METR.
<https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/>

</div>

<div id="ref-doi:10.1162/neco.2006.18.7.1527" class="csl-entry">

Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A Fast Learning
Algorithm for Deep Belief Nets. *Neural Computation*, *18*(7),
1527–1554. <https://doi.org/10.1162/neco.2006.18.7.1527>

</div>

<div id="ref-hf2026incident" class="csl-entry">

Hugging Face. (2026, July 16). *Security incident disclosure: July
2026*. <https://huggingface.co/blog/security-incident-july-2026>

</div>

<div id="ref-doi:10.48550/arXiv.2503.23674" class="csl-entry">

Jones, C. R., & Bergen, B. K. (2025). *Large Language Models Pass the
Turing Test* (Version 1). arXiv.
<https://doi.org/10.48550/arxiv.2503.23674>

</div>

<div id="ref-lighthill1973" class="csl-entry">

Lighthill, J. (1973). *Artificial intelligence: A general survey*
\[Artificial Intelligence: A Paper Symposium\]. Science Research
Council.

</div>

<div id="ref-doi:10.1007/BF02478259" class="csl-entry">

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas
immanent in nervous activity. *The Bulletin of Mathematical Biophysics*,
*5*(4), 115–133. <https://doi.org/10.1007/bf02478259>

</div>

<div id="ref-pmid:38746668" class="csl-entry">

Meng, X., Yan, X., Zhang, K., Liu, D., Cui, X., Yang, Y., Zhang, M.,
Cao, C., Wang, J., Wang, X., Gao, J., Wang, Y.-G.-S., Ji, J.-M., Qiu,
Z., Li, M., Qian, C., Guo, T., Ma, S., Wang, Z., … Tang, Y.-D. (2024).
The application of large language models in medicine: A scoping review.
*iScience*, *27*(5), 109713.
<https://doi.org/10.1016/j.isci.2024.109713>

</div>

<div id="ref-doi:10.1093/bib/bbx044" class="csl-entry">

Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2017). Deep
learning for healthcare: Review, opportunities and challenges.
*Briefings in Bioinformatics*, *19*(6), 1236–1246.
<https://doi.org/10.1093/bib/bbx044>

</div>

<div id="ref-openai2026hf" class="csl-entry">

OpenAI. (2026, August 26). *The hugging face incident and the road
ahead*.
<https://openai.com/index/hugging-face-incident-and-the-road-ahead/>

</div>

<div id="ref-doi:10.1037/h0042519" class="csl-entry">

Rosenblatt, F. (1958). The perceptron: A probabilistic model for
information storage and organization in the brain. *Psychological
Review*, *65*(6), 386–408. <https://doi.org/10.1037/h0042519>

</div>

<div id="ref-doi:10.1016/j.jacr.2018.03.046" class="csl-entry">

Schier, R. (2018). Artificial Intelligence and the Practice of
Radiology: An Alternative View. *Journal of the American College of
Radiology*, *15*(7), 1004–1007.
<https://doi.org/10.1016/j.jacr.2018.03.046>

</div>

<div id="ref-doi:10.1093/jnci/djx113" class="csl-entry">

Schmidt, C. (2017). M. D. Anderson Breaks With IBM Watson, Raising
Questions About Artificial Intelligence in Oncology. *JNCI: Journal of
the National Cancer Institute*, *109*(5).
<https://doi.org/10.1093/jnci/djx113>

</div>

<div id="ref-scspgallup2026" class="csl-entry">

Special Competitive Studies Project, & Gallup. (2026, September 16).
*Looking to the future: American priorities for AI in 2026*.
<https://www.prnewswire.com/news-releases/new-scsp-gallup-report-americans-agree-on-how-the-country-should-prepare-for-ai-302880594.html>

</div>

<div id="ref-guardian2026explosion" class="csl-entry">

The Guardian. (2026, September 28). *AI godfathers warn of runaway
“intelligence explosion”*.
<https://www.theguardian.com/technology/2026/sep/28/ai-godfathers-warn-of-runaway-intelligence-explosion>

</div>

<div id="ref-doi:10.1093/mind/LIX.236.433" class="csl-entry">

Turing, A. M. (1950). Computing Machinery and Intelligence. *Mind*,
*LIX*(236), 433–460. <https://doi.org/10.1093/mind/lix.236.433>

</div>

<div id="ref-wired2026gebru" class="csl-entry">

WIRED. (2026). *Timnit gebru believes there is no “existential threat”
from AI*.
<https://www.wired.com/story/the-big-interview-podcast-timnit-gebru/>

</div>

<div id="ref-doi:10.1001/jamainternmed.2021.2626" class="csl-entry">

Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J.,
DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C.,
Ghous, M., & Singh, K. (2021). External Validation of a Widely
Implemented Proprietary Sepsis Prediction Model in Hospitalized
Patients. *JAMA Internal Medicine*, *181*(8), 1065.
<https://doi.org/10.1001/jamainternmed.2021.2626>

</div>

<div id="ref-pmid:480542" class="csl-entry">

Yu, V. L., Fagan, L. M., Wraith, S. M., Clancey, W. J., Scott, A. C.,
Hannigan, J., Blum, R. L., Buchanan, B. G., & Cohen, S. N. (1979).
Antimicrobial selection by a computer. A blinded evaluation by
infectious diseases experts. *JAMA*, *242*(12), 1279–1282.
<https://www.ncbi.nlm.nih.gov/pubmed/480542>

</div>

</div>

<div class="notes">

The full references for the talk. The slides are posted on the course
site, and every source in the deck links from here.

</div>

[^1]: Geoffrey Hinton: computer scientist, University of Toronto; 2024
    Nobel Prize in Physics.

[^2]: Yoshua Bengio: computer scientist, Université de Montréal; founded
    the Mila AI institute; 2018 Turing Award.

[^3]: Volker Türk: UN High Commissioner for Human Rights since 2022.

[^4]: Timnit Gebru: computer scientist; founder of the DAIR research
    institute; formerly co-led Google’s Ethical AI team.
