From Simple Models to Deep Networks

Exploring the Path from Traditional Machine Learning to Neural Networks, One Neuron at a Time

Sean Davis, MD, PhD

University of Colorado School of Medicine

November 6, 2024

Deep Learning in Healthcare and Biomedical Research

Deep Learning in Healthcare

Esteva et al. (2019)

Deep learning in medical imaging

Litjens et al. (2017)

Opportunities and obstacles

Ching et al. (2018)

Neural network zoo

Traditional Machine Learning to Deep Learning

Traditional Machine Learning

• Linear regression.
• Logistic regression.
• Decision trees.
• Random forests.
• Support vector machines.
• Gradient boosting.

Neural Networks

• Perceptrons.
• Multilayer perceptrons.
• Convolutional neural networks.
• Recurrent neural networks.
• Transformers.
• GANs.
• Autoencoders.

Traditional machine learning approaches

Deep learning from the ground up

Linear regression as a neuron

Linear regression

\(Y = \beta_0 + \beta_1 X\)

Linear regression as a neuron

\(F(x) = b + w^T X\)

For one predictor, this reduces to:

\(F(x) = b + w_1 X\)

Linear regression

Linear regression

• Input: \(X\)
• Output: \(Y\)
• Model: \(Y = \beta_0 + \beta_1 X\)
• Loss: Mean squared error
• Optimizer: least squares

Neuron

• Input: \(X\)
• Output: \(Y\)
• Model: \(Y = b + w_1 X\)
• Loss: Mean squared error
• Optimizer: Gradient descent

Image classification

The MNIST (modified NIST) database contains 60,000 training images and 10,000 testing images.

Image classification

Image classification

Image classification

Image classification

Visualization of weights associated with each class. Blue is “positive” and red is “negative”.

Hands on

Neural network playground

Tensorflow playground tutorial

• https://cloud.google.com/blog/products/ai-machine-learning/understanding-neural-networks-with-tensorflow-playground

References

Ching, Travers, Daniel S. Himmelstein, Brett K. Beaulieu-Jones, Alexandr A. Kalinin, Brian T. Do, Gregory P. Way, Enrico Ferrero, et al. 2018. “Opportunities and Obstacles for Deep Learning in Biology and Medicine.” Journal of The Royal Society Interface 15 (141): 20170387. https://doi.org/10.1098/rsif.2017.0387.

Esteva, Andre, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. 2019. “A Guide to Deep Learning in Healthcare.” Nature Medicine 25 (1): 24–29. https://doi.org/10.1038/s41591-018-0316-z.

Leijnen, Fjodor van Veen, Stefan. 2016. “The Neural Network Zoo.” The Asimov Institute. https://www.asimovinstitute.org/neural-network-zoo/.

Litjens, Geert, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A. W. M. van der Laak, Bram van Ginneken, and Clara I. Sánchez. 2017. “A Survey on Deep Learning in Medical Image Analysis.” Medical Image Analysis 42 (December): 60–88. https://doi.org/10.1016/j.media.2017.07.005.