What we will cover

  • AI, ML, DL, GenAI: how the four fit inside each other
  • One model, start to finish: the eGFR, fitting a model to data, adding a predictor
  • The fitting loop, and two questions that sort any model
  • What goes wrong: models that are too flexible, neural networks, and why a score on cases a model has already seen proves nothing
  • Back to the clinic: what this changes on the wards

Questions along the way are live: answer on your phone or laptop. Thursday starts with the chatbot half of this story.

Four words, each inside the one before

Four nested circles. The largest is artificial intelligence: a computer doing a task that seems to need judgment, such as MYCIN in the 1970s, whose rules were written by hand. Inside it is machine learning: the rules are learned from data, such as the eGFR equation. Inside that is deep learning: machine learning with many layers, a neural network, such as a model that reads chest X-rays. The smallest is generative AI: deep learning that writes new text or makes images, such as a chatbot.

Each is one kind of the one around it. Yesterday we saw how we got here. Today: how the models inside these circles are made.

Every model has the same setup

Yi = f ( Xi ) + εi Target the value we want to predict Model a function: turns the inputs into a prediction Predictors the inputs we know, for person i Error what the model misses: target minus prediction

Loss function One number that adds up the errors across everyone: the sum of squared errors, or the average miss. Big errors, big loss. Fitting means choosing the model that makes the loss as small as it can get.

Predict weight from height

Made-up data for 40 adults.

What is missing? What else would help predict weight? In other words, what additional predictors should we consider?

weight = a + b × height + error

a: the intercept. b: the slope, kg per extra centimeter.

What you just did is the fitting loop: guess the dials, score the misses with the loss, adjust, repeat.

Part 1 The number nobody measured

Pre-op clinic, surgery sub-internship

A 62-year-old woman is booked for a laparoscopic cholecystectomy. Her basic metabolic panel is back.

The attending asks: are her kidneys all right for the contrast CT?

You say: “eGFR 57, stage 3a, mildly reduced. Probably fine with hydration.”

Basic metabolic panel · collected 07:12
Sodium 139 136–145
Potassium 4.1 3.5–5.1
Creatinine 1.10 0.50–1.10
eGFR (CKD-EPI 2021) 57 L >60
Units: mmol/L; creatinine mg/dL; eGFR mL/min/1.73 m². Illustrative panel, made up for this course. Not a real patient.

Where does the 57 come from?

Her report says eGFR 57. Where does the 57 come from?

  • The analyzer measured it in her blood sample
  • It was measured from a timed urine collection
  • An equation calculated it from her creatinine, age, and sex

Nobody measured her GFR

  • Measuring GFR means infusing a marker the kidney filters and does nothing else with, such as iohexol, and timing how fast it clears.
  • That takes hours. It is rarely done in routine care.
  • The 57 was computed from creatinine, age, and sex.
  • The equation was learned from data, not from physiology. A few thousand people did have their GFR measured. A computer was given their creatinine, age, and sex, and one instruction: find the formula that best predicts the measured number.

The eGFR is a model. Remember her: 45 kg, and a year of muscle loss.

Fit the curve yourself

Illustrative data, not the CKD-EPI data. A simpler curve than the real eGFR.

GFR = a × creatinine−b

a: the GFR when creatinine is 1.0. b: how fast GFR falls as creatinine rises.

Fitting means a computer turns these dials until the error is as small as it can get.

● one made-up person · thin line: how far the curve misses them · thick line: the curve your dials make. Generated in your browser.

Add a predictor

Illustrative data, made up. Women are built to have a lower GFR at the same creatinine.

Dots: teal men, terracotta women. Thin lines: the miss for each person. The fit makes the RSS as small as it can, on a log scale.

Fitting with a second predictor is the same job: find the numbers that make the misses smallest.

Part 1, continued The loop every model comes from

Six words for the rest of the course

Word Meaning In the eGFR
Predictors the inputs creatinine, age, sex
Response what is being predicted measured GFR
Training set the cases used to fit the equation 8,254 adults in 10 studies, mostly research participants; 31.5% Black
Validation set cases held back during fitting, to choose between versions see the note below
Test set cases held back to check it once, at the end 4,050 people in 12 other studies; the paper calls them its “validation data set”
Model the equation once fit eGFR = f(creatinine, age, sex)

f stands for function: a rule that turns inputs into an output. Fit means a computer tried numbers in the equation until its predictions were as close to the measured values as they could get (Inker et al., 2021).

Words vary by community. Statistics says response or outcome; machine learning says target or label (slide 4 used target). Medical papers often say validation set for what machine learning calls the test set. Ask what the held-back cases were used for, not what they are called.

Split the data before you fit

Diagram. One table, one row per patient, with columns creatinine, age and sex as predictors and measured GFR as the target. The rows are cut into three groups. The training set, about 70 percent, goes to fit: the loop turns the dials on these cases only. The validation set, about 15 percent, goes to choose: when to stop, and which model to keep; a dashed arrow shows it checking the loop. The test set, about 15 percent, carries a padlock and goes to check once, at the end: the number you report. Footnote: split by patient, before any fitting; shares are typical, not a rule.

The eGFR’s test set wasn’t a random slice of the same table: it was 4,050 people from 12 other studies. That’s a harder test, and a better one.

One equation, three generations

Equation Learned from Where it failed, and for whom
MDRD, 1999 1,070 patients who already had kidney disease (558 more held back to test it) Values too low at higher GFR, so healthy people were labelled as diseased (Levey et al., 1999; Levey et al., 2009)
CKD-EPI, 2009 8,254 people in 10 studies Included a race term. Even with it, the equation overestimated measured GFR in Black adults (median 3.7 mL/min/1.73 m²). An overestimate makes kidney disease look milder than it is (Inker et al., 2021)
CKD-EPI, 2021 the same 8,254, refit without race Now a few mL/min too low in Black adults and too high in others. A version that adds cystatin C, a second blood marker that does not depend on muscle, is more accurate (Delgado et al., 2021; Inker et al., 2021)

The training population is part of the model,
as much as the equation is.

The fitting loop

Diagram. A loop of four steps: 1, guess the dials; 2, predict every training case; 3, score, adding up the misses into the loss; 4, adjust, turning the dials to lower it; then back to step 1, repeating while the loss on validation cases improves. A small chart shows two curves lap by lap: the training loss keeps falling, while the validation loss falls, bottoms out, and rises, with a line marking stop here. When validation says stop, an arrow exits the loop to step 5: test, once, on cases the loop never saw. Below, a strip: training set, then the fitting loop checked on a validation set, then a test set used once, giving a model f; a new case goes through f to give a prediction.

You ran this loop by hand with the height and weight sliders. Every model in this course is fit this way: only the data and the number of dials change.

Two questions sort any model

A flow chart. Question 1: does the training data contain the answer? If no, the model is unsupervised: it finds groups, such as breast cancer subtypes. If yes, question 2: what does it output? A number means regression, such as an eGFR or an ejection fraction, and the model's errors are distances from the line. A category means classification, such as benign nevus or melanoma, and its errors are on the wrong side of a boundary.

A regression is wrong by an amount. A classification is wrong by a kind: a miss, or a false alarm. A miss costs the patient who was missed. A false alarm costs the staff who answer it, and the patient worked up for nothing.

A model watches a heart ultrasound

A model watches a heart ultrasound video and reports an ejection fraction of 55%. What kind of model is it?

  • Regression: it gives a number
  • Classification: it gives a category
  • Unsupervised: it finds groups, with no answer column

A model looks at a skin photo

A model looks at a photo of a skin lesion and answers: benign nevus or melanoma. What kind of model is it?

  • Regression: it gives a number
  • Classification: it gives a category
  • Unsupervised: it finds groups, with no answer column

Four fields, the same two questions

Field The model reads It gives Type What the authors reported
Cardiology a heart ultrasound video ejection fraction Regression average error 4.1 points at the same hospital, 6.0 at another health system (Ouyang et al., 2020)
Pediatrics a hand X-ray bone age in months Regression the best of 105 entries was off by 4.2 months on average, on 200 test X-rays (Halabi et al., 2018)
Dermatology a skin photo benign or malignant Classification on par with 21 dermatologists, on biopsy-proven images (Esteva et al., 2017)
Pathology a lymph node slide cancer present or absent Classification best entry AUC 0.994; the entries ranged from 0.556 to 0.994 (Ehteshami Bejnordi et al., 2017)

More of the same: an ECG rhythm classifier that matched cardiologists (Hannun et al., 2019); breast tumors grouped by gene expression, with no diagnoses attached, which is unsupervised (Perou et al., 2000); and the diabetic retinopathy system from Tuesday (Abràmoff et al., 2018).

Too simple, too flexible

A model overfits when a simpler model would do worse on the training cases but better on new cases from the same population (Mitchell, 1997).

● learns from · ○ held back for testing · dashed: the real pattern. Made-up data. Bias: error of a too-simple model. Variance: error of a too-flexible one.

A neural network: the same loop, more dials

Illustrative. Every line is a dial. This toy network has ; the networks behind the examples have far more.

  • The inputs can be the raw pixels of an image. Nobody has to write down what to look for.
  • Each unit adds up what it receives, with its own dials, and passes the result on.
  • Fitting is the same loop: turn every dial until the misses on the training cases get smaller.

Simple models are easier to check

Simple: the eGFR equation

  • A handful of numbers. You can read each one.
  • You can check them against what you know: higher creatinine, lower eGFR.
  • A few thousand people were enough to fit it.
  • It can only follow the shape its equation allows.

Flexible: a neural network

  • Thousands to billions of numbers. Nobody reads them one by one.
  • It can fit complex patterns: an image, a sentence.
  • It is harder to explain why it gave one answer.
  • It needs much more data, or it fits the noise, as in the slider.

More flexible: more patterns it can fit,
less you can explain, more data it needs.

A model is interpretable when a person can read why it gave its answer.

Cases it has already seen prove nothing

Fit a formula to 8,254 people, then report how well it predicts those same people, and you have measured how well it memorized them.

A separate test set is the least you should accept.
A number with no test set is not a number yet.

The usual rule is three sets: a training set to fit the model, a validation set to choose between models, and a test set, used once, at the end.

You saw this sepsis model yesterday

Here is why it belongs in today’s talk: it is the same failure, on new patients. A vendor sepsis model, in use at hundreds of US hospitals in 2021, was tested at one academic center on 38,455 hospitalizations (2018 to 2019), by people who had not built it (Wong et al., 2021).

  • About 7 in 100 admissions had sepsis.
  • Area under the curve 0.63 (0.5 is chance; 1.0 is perfect).
  • It missed 67% of patients with sepsis.
  • It alerted on 18% of all admissions, so most alerts were for patients without sepsis.

One grid of 200 squares, one per admission. The first 36 squares are colored for an alert, the rest for no alert. Thirteen squares carry a dot for a patient with sepsis: 4 in alerted squares and 9 in squares with no alert.

Redrawn from the counts reported by Wong et al. (2021), scaled to 200 admissions.

Testing on patients from a different setting than the model was built on is external validation. Of the 13 with sepsis, 9 sat in a no-alert box. No alert does not mean no sepsis.

The published number is the optimistic one

A model’s performance on the cases it was built from is usually the optimistic number.

Ask what it was trained on.
Ask what it was tested on, and by whom.
If the training set and the test set are the same cases, the score tells you nothing yet.

Better still, a hospital tests the model on its own past patients before go-live (localization), then keeps checking that it still works (monitoring).

A language model is the same loop, with a different table

the patient was started on empiric vancomycin

Predictors: the words so far Truth: the next word
the patient
the patient was
the patient was started
the patient was started on
the patient was started on empiric
the patient was started on empiric vancomycin

Nobody labels anything. Every sentence ever written is already a table like this.

Part 3 What this changes for you

Back to the pre-op clinic

  • She weighs 45 kg and has lost muscle over a year of illness.
  • Creatinine comes from muscle.
  • Her creatinine of 1.10 is a normal number for a normal amount of muscle. She does not have a normal amount of muscle.
  • So her true GFR is probably lower than 57.

Name her model’s predictors, response, and training population.

What do you now say to the attending?

“The 57 is estimated from her creatinine. With her low muscle mass it probably overstates her kidney function. If the decision depends on it, I would ask for a cystatin C-based eGFR.”

It matters most for a renally dosed drug, or if her true value could be near your radiology department’s contrast threshold. A national task force recommends wider use of cystatin C, especially to confirm the estimate (Delgado et al., 2021).

Nobody told her a model was involved. Whether someone should have is a question for the patient communication session.

On the wards

  • An alert fires, or stays quiet. No alert does not mean no sepsis. If you silence one, be able to say why.
  • A tool comes with an accuracy number. Ask what it was tested on, and by whom. A score on the cases it was built from is the optimistic one.
  • A tool drafted your note or message. Read all of it before you sign. You are responsible for what it says.
  • Patient information. Not in any chatbot, unless your hospital’s privacy office has approved that tool in writing.
  • Documentation. What to record when a tool helped depends on your site’s policy. Ask your supervisor.

Ask what it was tested on,
and by whom.

The standing question

What does this change about how you’ll practice?

Write it down before you close this deck. Then one more: the next time someone shows you a model’s accuracy number, what will you ask in the next two minutes? For example: what was it tested on, and by whom?

Thursday: we start with the chatbot, the same loop with words. Then we open ChatGPT Edu together and use it to set up your own learning goals and journal.

Summary

  • AI contains machine learning, which contains deep learning, which contains generative AI.
  • The eGFR is a model learned from data: creatinine, age, and sex in; a stand-in for measured GFR out; 8,254 adults as the population it was fit to. Fitting chose its numbers to make the error as small as possible.
  • Supervised has the answer in the training data; unsupervised does not. Classification gives a category; regression gives a number.
  • A model that is too flexible fits the noise: it overfits. A neural network is the same loop with far more dials, so it needs more data, and a test set matters even more.
  • A score on the training data describes the fit. Ask for the test set, and who ran it.
  • Thursday: a language model is the same loop, with words.

Further exploration

Try: overfitting you can see

TensorFlow Playground, 10 minutes, no account. Open step 1 (too simple), step 2 (in between) and step 3 (too flexible). At about 500 epochs, compare training loss with test loss.

Read: when the patients change

The clinician and dataset shift in AI, NEJM, 2021. Why models fail on new kinds of patients.

References

Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digital Medicine, 1, 39. https://doi.org/10.1038/s41746-018-0040-6
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). https://arxiv.org/abs/2005.14165
Delgado, C., Baweja, M., Crews, D. C., Eneanya, N. D., Gadegbeku, C. A., Inker, L. A., Mendu, M. L., Miller, W. G., Moxey-Mims, M. M., Roberts, G. V., St Peter, W. L., Warfield, C., & Powe, N. R. (2021). A Unifying Approach for GFR Estimation: Recommendations of the NKF-ASN Task Force on Reassessing the Inclusion of Race in Diagnosing Kidney Disease. American Journal of Kidney Diseases : The Official Journal of the National Kidney Foundation, 79(2), 268–288.e1. https://doi.org/10.1053/j.ajkd.2021.08.003
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., Ginneken, B. van, Karssemeijer, N., Litjens, G., Laak, J. A. W. M. van der, Hermsen, M. and, Manson, Q. F., Balkenhol, M., Geessink, O., Stathonikos, N., Dijk, M. C. van, Bult, P., Beca, F., Beck, A. H., Wang, D., Khosla, A., Gargeya, R., … Venâncio, R. (2017). Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA, 318(22), 2199–2210. https://doi.org/10.1001/jama.2017.14585
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. https://doi.org/10.1038/nature21056
Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S. (2021). The Clinician and Dataset Shift in Artificial Intelligence. The New England Journal of Medicine, 385(3), 283–286. https://doi.org/10.1056/nejmc2104626
Halabi, S. S., Prevedello, L. M., Kalpathy-Cramer, J., Mamonov, A. B., Bilbily, A., Cicero, M., Pan, I., Pereira, L. A., Sousa, R. T., Abdala, N., Kitamura, F. C., Thodberg, H. H., Chen, L., Shih, G., Andriole, K., Kohli, M. D., Erickson, B. J., & Flanders, A. E. (2018). The RSNA Pediatric Bone Age Machine Learning Challenge. Radiology, 290(2), 498–503. https://doi.org/10.1148/radiol.2018180736
Hannun, A. Y., Rajpurkar, P., Haghpanahi, M., Tison, G. H., Bourn, C., Turakhia, M. P., & Ng, A. Y. (2019). Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nature Medicine, 25(1), 65–69. https://doi.org/10.1038/s41591-018-0268-3
Inker, L. A., Eneanya, N. D., Coresh, J., Tighiouart, H., Wang, D., Sang, Y., Crews, D. C., Doria, A., Estrella, M. M., Froissart, M., Grams, M. E., Greene, T., Grubb, A., Gudnason, V., Gutiérrez, O. M., Kalil, R., Karger, A. B., Mauer, M., Navis, G., … Levey, A. S. and. (2021). New Creatinine- and Cystatin C-Based Equations to Estimate GFR without Race. The New England Journal of Medicine, 385(19), 1737–1749. https://doi.org/10.1056/nejmoa2102953
Levey, A. S., Bosch, J. P., Lewis, J. B., Greene, T., Rogers, N., & Roth, D. (1999). A more accurate method to estimate glomerular filtration rate from serum creatinine: A new prediction equation. Modification of Diet in Renal Disease Study Group. Annals of Internal Medicine, 130(6), 461–470. https://doi.org/10.7326/0003-4819-130-6-199903160-00002
Levey, A. S., Stevens, L. A., Schmid, C. H., Zhang, Y. L., Castro, A. F., Feldman, H. I., Kusek, J. W., Eggers, P., Van Lente, F., Greene, T., & Coresh, J. and. (2009). A new equation to estimate glomerular filtration rate. Annals of Internal Medicine, 150(9), 604–612. https://doi.org/10.7326/0003-4819-150-9-200905050-00006
Mitchell, T. M. (1997). Machine learning. McGraw-Hill.
Ouyang, D., He, B., Ghorbani, A., Yuan, N., Ebinger, J., Langlotz, C. P., Heidenreich, P. A., Harrington, R. A., Liang, D. H., Ashley, E. A., & Zou, J. Y. (2020). Video-based AI for beat-to-beat assessment of cardiac function. Nature, 580(7802), 252–256. https://doi.org/10.1038/s41586-020-2145-8
Perou, C. M., Sørlie, T., Eisen, M. B., Rijn, M. van de, Jeffrey, S. S., Rees, C. A., Pollack, J. R., Ross, D. T., Johnsen, H., Akslen, L. A., Fluge, O., Pergamenschikov, A., Williams, C., Zhu, S. X., Lønning, P. E., Børresen-Dale, A. L., Brown, P. O., & Botstein, D. (2000). Molecular portraits of human breast tumours. Nature, 406(6797), 747–752. https://doi.org/10.1038/35021093
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., Ghous, M., & Singh, K. (2021). External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Internal Medicine, 181(8), 1065–1070. https://doi.org/10.1001/jamainternmed.2021.2626

Questions