The History of Artificial Intelligence and Machine Learning

Sean Davis, MD, PhD

Thursday, September 19, 2024

Introduction

  • Artificial Intelligence (AI) and Machine Learning (ML) have a rich history
  • From early concepts to modern applications
  • This presentation covers key milestones and breakthroughs

Early Beginnings (1940s-1950s)

  • 1936: Turing and the Computable Numbers (Turing 1936)
  • 1943: McCulloch and Pitts create a computational model for neural networks
  • 1950: Alan Turing proposes the Turing Test (Turing 1950) and Paper
  • 1956: Dartmouth Conference coins the term “Artificial Intelligence”

“I, Robot” by Isaac Asimov

  • Published in 1950 by Gnome Press
  • Collection of nine science fiction short stories
  • Originally appeared in super-science fiction magazines (1940-1950)
  • Introduced the concept of positronic robots and the Three Laws of Robotics

Asimov (1950)

The Three Laws of Robotics

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

Asimov later added the Zeroth Law that superseded the others:

  1. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

“I, Robot”’s Influence on Modern AI

  • Sparked discussions on machine ethics and AI safety
  • Influenced researchers to consider ethical implications of AI development
  • Concept of “friendly AI” draws parallels to Asimov’s laws
  • Challenges presented in stories mirror real-world AI alignment problems

1950: The Turing Test

Who’s the real human?

The Golden Years (1956-1974)

  • Development of early AI programs
  • 1957: Frank Rosenblatt develops the Perceptron
  • 1964: ELIZA, one of the first chatbots, is created by Joseph Weizenbaum

1964: ELIZA, one of the first chatbots

Although ELIZA was limited in terms of actual understanding, it marked an important milestone in the development of AI and human-computer interaction, showing how conversation-based interfaces could influence the perception of intelligence.

Expert Systems Era (1970s-1980s)

Expert Systems in Artificial Intelligence

What is an Expert System?

  • Definition: Computer program that emulates decision-making ability of a human expert
  • Key Components:
    1. Knowledge Base: Contains domain-specific information and rules
    2. Inference Engine: Applies rules to the knowledge to derive new information
    3. User Interface: Allows non-expert users to interact with the system
  • Characteristics:
    • Solves complex problems by reasoning through bodies of knowledge
    • Separates domain knowledge from the reasoning mechanism
    • Can explain its decisions and reasoning

What is an Expert System?

  • Applications:
    • Medical diagnosis (e.g., MYCIN)
    • Financial planning
    • Manufacturing process control
    • Scientific analysis
  • Advantages:
    • Consistent and accurate decisions
    • Preservation of expert knowledge
    • Ability to handle complex scenarios
  • Limitations:
    • Limited to specific domains (narrow AI)
    • Difficulty in capturing tacit knowledge
    • May struggle with unusual or unprecedented situations

Example Expert system: MYCIN

  • Developed in the early 1970s at Stanford University
  • One of the first rule-based expert systems in medicine
  • Purpose: Assist physicians in diagnosing and treating bacterial infections
  • Focused on bloodstream infections (bacteremia and meningitis)
  • Named after antibiotics (many of which end in “-mycin”)

MYCIN: Key Features and Functionality

  • Rule-based system with approximately 600 rules
  • Used backward chaining inference engine
  • Incorporated certainty factors to handle uncertainty
  • Asked users a series of yes/no questions about symptoms and test results
  • Provided diagnosis recommendations and suggested antibiotic treatments
  • Explained its reasoning process to user

MYCIN: Impact and Legacy

  • Never used in clinical practice due to ethical and legal concerns
  • Achieved performance comparable to human experts in its domain
  • Pioneered several important concepts in AI and expert systems:
    • Separation of knowledge base from inference engine
    • Explanation of reasoning
    • Handling of uncertainty
  • Influenced development of subsequent expert systems and clinical decision support tools
  • Demonstrated potential of AI in healthcare, paving way for modern medical AI applications

AI Winter (1974-1980 and 1987-1993)

  • Periods of reduced funding and interest in AI
  • Overpromising and underdelivering led to skepticism
  • Shift towards more practical, focused applications

Revival of Machine Learning (1990s)

  • Increased focus on data-driven approaches
  • 1997: IBM’s Deep Blue defeats world chess champion Garry Kasparov
  • Growing interest in neural networks and statistical methods

“Technological Triggers for the Rise of ML and AI in the 2000s”

Increased Computational Power

  • Advancement:
    • Rapid growth of CPUs and the emergence of GPUs (Graphics Processing Units).
  • Impact:
    • Enabled the training of deeper neural networks essential for various AI tasks.

Big Data

  • Advancement:
    • Explosion of digital data from the internet, social media, and sensors.
  • Impact:
    • Facilitated the development of accurate models as ML algorithms require substantial data to learn effectively.

Open Source Frameworks and Libraries

  • Advancement:
    • Emergence of libraries like TensorFlow (2015), Keras (2015), and Scikit-learn (2007).
  • Impact:
    • Lowered the barrier for AI development, allowing more practitioners to innovate in the field.

Advances in Algorithms

  • Advancement:
    • Research in new algorithms, such as support vector machines and deep learning architectures.
  • Impact:
    • Improved AI performance across applications, particularly in image and speech recognition.

Cloud Computing

  • Advancement:
    • Rise of cloud computing platforms (e.g., AWS, Google Cloud, Microsoft Azure).
  • Impact:
    • Provided scalable resources for storage and computation, enabling extensive ML experimentation.

Collaborative Research and Knowledge Sharing

  • Advancement:
    • Increased collaboration and sharing of findings through conferences and online platforms.
  • Impact:
    • Accelerated innovation in AI and ML as researchers built upon each other’s work.

Investment and Interest from Industry

  • Advancement:
    • Growing interest and investment from tech giants and startups in AI technologies.
  • Impact:
    • Led to the development of practical applications and commercial products, driving further research.

The 2000s: The Rise of Big Data and Deep Learning

Deep Learning Resurgence (2006)

  • Milestone: Geoffrey Hinton and team introduced “deep belief networks.”
  • Impact: Marked the resurgence of deep learning and laid the foundation for modern AI applications, especially in image and speech recognition.

AlexNet Wins ImageNet Competition (2012)

  • Milestone: Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton’s deep neural network (AlexNet) won the ImageNet competition.
  • Impact: Showcased the power of convolutional neural networks (CNNs) and triggered widespread adoption in computer vision tasks.

IBM Watson Wins Jeopardy! (2011)

  • Milestone: IBM Watson defeated champions Ken Jennings and Brad Rutter on Jeopardy!.
  • Impact: Demonstrated AI’s ability to process and understand natural language, leading to applications in healthcare, finance, and customer service.

MD Anderson sets Watson aside (2017-2018)

See Herper (n.d.)

Generative Adversarial Networks (GANs) (2014)

  • Milestone: Ian Goodfellow introduced GANs, a model where two neural networks compete to generate realistic data.
  • Impact: Revolutionized image generation and unsupervised learning, powering innovations like deepfakes and AI-generated art.

AlphaGo Defeats World Champion (2016)

  • Milestone: Google DeepMind’s AlphaGo defeated Go champion Lee Sedol.
  • Impact: Showcased the capability of reinforcement learning and deep neural networks in mastering complex strategic games.

Transformer Architecture (2017)

  • Milestone: Vaswani et al. introduced the Transformer model, revolutionizing natural language processing.
  • Impact: Laid the groundwork for state-of-the-art NLP models like BERT and GPT, transforming language understanding and generation.

AI and ML in Social Media (2010s)

  • Milestone: Social media platforms adopted AI for content recommendation and moderation.
  • Impact: Enhanced user engagement and experience, but also raised concerns about echo chambers, misinformation, and algorithmic bias.

AlphaFold Solves Protein Folding (2020)

  • Milestone: DeepMind’s AlphaFold achieved breakthrough accuracy in predicting protein structures.
  • Impact: Solved a 50-year-old challenge in biology, opening new doors in drug discovery and molecular biology.

GPT-4 and Large Language Models (2023)

  • Milestone: OpenAI’s GPT-4 showcased the potential of large-scale language models for complex, nuanced language understanding.
  • Impact: Accelerated the development of AI-driven content creation and enhanced human-computer interaction.

Future Directions

  • Artificial General Intelligence (AGI) research
  • Quantum computing and AI
  • Neuromorphic computing
  • Human-AI collaboration

For deeper dive into the history, see (Norman 2024).

Challenges and Opportunities

  • Ethical AI development
  • AI governance and regulation
  • Addressing AI bias and fairness
  • Balancing innovation with responsible development

References

Asimov, Isaac. 1950. I, Robot. Bantam hardcover ed. (2). New York: Bantam Books.
Herper, Matthew. n.d. “MD Anderson Benches IBM Watson In Setback For Artificial Intelligence In Medicine.” Forbes. Accessed October 2, 2024. https://www.forbes.com/sites/matthewherper/2017/02/19/md-anderson-benches-ibm-watson-in-setback-for-artificial-intelligence-in-medicine/.
Norman, Jeremy. 2024. “History of Information.” https://www.historyofinformation.com/?cat=71.
Turing, Alan. 1936. “On Computable Numbers, with an Application to the Entscheidungsproblem.” https://www.abelard.org/turpap2/tp2-ie.asp.
———. 1950. “Computing Machinery and Intelligence.” Mind 59 (October): 433–60. https://doi.org/10.1093/mind/lix.236.433.