The History of Artificial Intelligence and Machine Learning

Sean Davis, MD, PhD

October 30, 2024

Introduction

• Artificial Intelligence (AI) and Machine Learning (ML) have a rich history
• From early concepts to modern applications
• This presentation covers key milestones and breakthroughs

Early Beginnings (1940s-1950s)

• 1936: Turing and the Computable Numbers (Turing 1936)
• 1943: McCulloch and Pitts create a computational model for neural networks
• 1950: Alan Turing proposes the Turing Test (Turing 1950) and Paper
• 1956: Dartmouth Conference coins the term “Artificial Intelligence”

1936: Turing and the Computable Numbers

Alan Turing’s 1936 paper, “On Computable Numbers, with an Application to the Entscheidungsproblem”, introduced the idea of computable numbers and the Universal Turing Machine, and laid the theoretical foundation for modern computing.

1943: McCulloch and Pitts Neuron

• Warren McCulloch (neurophysiologist) and Walter Pitts (logician)

1943: McCulloch and Pitts Neuron

• Warren McCulloch (neurophysiologist) and Walter Pitts (logician)

1950: The Turing Test

Who’s the real human?

“I, Robot” by Isaac Asimov

• Published in 1950 by Gnome Press
• Collection of nine science fiction short stories
• Originally appeared in super-science fiction magazines (1940-1950)
• Introduced the concept of positronic robots and the Three Laws of Robotics

Asimov (1950)

The Three Laws of Robotics

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

Asimov later added the Zeroth Law that superseded the others:

0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

These laws have become a cornerstone in discussions about AI ethics and safety.

“I, Robot”’s Influence on Modern AI

• Sparked discussions on machine ethics and AI safety
• Influenced researchers to consider ethical implications of AI development
• Concept of “friendly AI” draws parallels to Asimov’s laws
• Challenges presented in stories mirror real-world AI alignment problems

1956: The Dartmouth Conference

The Dartmouth Conference, held in the summer of 1956, is considered the birthplace of artificial intelligence. The conference was organized by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon, who invited researchers to discuss the potential of creating machines that could simulate human intelligence.

Marvin Minsky, Claude Shannon, Ray Solomonoff and other scientists at the Dartmouth Summer Research Project on Artificial Intelligence (Photo: Margaret Minsky).

The Golden Years (1956-1974)

• Development of early AI programs
• 1957: Frank Rosenblatt develops the Perceptron
• 1964: ELIZA, one of the first chatbots, is created by Joseph Weizenbaum

1957: The Perceptron

1964: ELIZA, one of the first chatbots

Although ELIZA was limited in terms of actual understanding, it marked an important milestone in the development of AI and human-computer interaction, showing how conversation-based interfaces could influence the perception of intelligence.

ELIZA was one of the earliest chatbots, developed in the mid-1960s by Joseph Weizenbaum, a computer scientist at MIT. The program was designed to simulate a conversation between a human and a machine, and it did so by using pattern matching and substitution methodology, a simple but effective form of natural language processing.

ELIZA’s most famous script was called “DOCTOR,” which mimicked a Rogerian psychotherapist. In this role, ELIZA would take users’ input and reflect their statements back to them in a way that encouraged further conversation. For example, if a user said, “I’m feeling sad today,” ELIZA might respond with, “Why do you think you’re feeling sad today?”

While ELIZA’s responses were largely superficial, many users were surprised at how human-like they seemed. Weizenbaum created ELIZA to demonstrate how easily people could attribute human-like understanding to a machine, even when its responses were formulaic. He was struck by how quickly people became emotionally attached to the program, despite knowing it was not genuinely intelligent.

Although ELIZA was limited in terms of actual understanding, it marked an important milestone in the development of AI and human-computer interaction, showing how conversation-based interfaces could influence the perception of intelligence.

Expert Systems Era (1970s-1980s)

What is an Expert System?

• Definition: Computer program that emulates decision-making ability of a human expert
• Key Components:
  1. Knowledge Base: Contains domain-specific information and rules
  2. Inference Engine: Applies rules to the knowledge to derive new information
  3. User Interface: Allows non-expert users to interact with the system
• Characteristics:
  • Solves complex problems by reasoning through bodies of knowledge
  • Separates domain knowledge from the reasoning mechanism
  • Can explain its decisions and reasoning

What is an Expert System?

• Applications:
  • Medical diagnosis (e.g., MYCIN)
  • Financial planning
  • Manufacturing process control
  • Scientific analysis
• Advantages:
  • Consistent and accurate decisions
  • Preservation of expert knowledge
  • Ability to handle complex scenarios
• Limitations:
  • Limited to specific domains (narrow AI)
  • Difficulty in capturing tacit knowledge
  • May struggle with unusual or unprecedented situations

Example Expert system: MYCIN

• Developed in the early 1970s at Stanford University
• One of the first rule-based expert systems in medicine
• Purpose: Assist physicians in diagnosing and treating bacterial infections
• Focused on bloodstream infections (bacteremia and meningitis)
• Named after antibiotics (many of which end in “-mycin”)

MYCIN: Key Features and Functionality

• Rule-based system with approximately 600 rules
• Used backward chaining inference engine
• Incorporated certainty factors to handle uncertainty
• Asked users a series of yes/no questions about symptoms and test results
• Provided diagnosis recommendations and suggested antibiotic treatments
• Explained its reasoning process to user

MYCIN: Impact and Legacy

• Never used in clinical practice due to ethical and legal concerns
• Achieved performance comparable to human experts in its domain
• Pioneered several important concepts in AI and expert systems:
  • Separation of knowledge base from inference engine
  • Explanation of reasoning
  • Handling of uncertainty
• Influenced development of subsequent expert systems and clinical decision support tools
• Demonstrated potential of AI in healthcare, paving way for modern medical AI applications

AI Winter (1974-1980 and 1987-1993)

• Periods of reduced funding and interest in AI
• Overpromising and underdelivering led to skepticism
• Shift towards more practical, focused applications

Revival of Machine Learning (1990s)

• Increased focus on data-driven approaches
• 1997: IBM’s Deep Blue defeats world chess champion Garry Kasparov
• Growing interest in neural networks and statistical methods

Technological Triggers for the Rise of ML and AI in the 2000s

Increased Computational Power

• Advancement:
  • Rapid growth of CPUs and the emergence of GPUs (Graphics Processing Units).
• Impact:
  • Enabled the training of deeper neural networks essential for various AI tasks.

Big Data

• Advancement:
  • Explosion of digital data from the internet, social media, and sensors.
• Impact:
  • Facilitated the development of accurate models as ML algorithms require substantial data to learn effectively.

Open Source Frameworks and Libraries

• Advancement:
  • Emergence of libraries like TensorFlow (2015), Keras (2015), and Scikit-learn (2007).
• Impact:
  • Lowered the barrier for AI development, allowing more practitioners to innovate in the field.

Advances in Algorithms

• Advancement:
  • Research in new algorithms, such as support vector machines and deep learning architectures.
• Impact:
  • Improved AI performance across applications, particularly in image and speech recognition.

Cloud Computing

• Advancement:
  • Rise of cloud computing platforms (e.g., AWS, Google Cloud, Microsoft Azure).
• Impact:
  • Provided scalable resources for storage and computation, enabling extensive ML experimentation.

Collaborative Research and Knowledge Sharing

• Advancement:
  • Increased collaboration and sharing of findings through conferences and online platforms.
• Impact:
  • Accelerated innovation in AI and ML as researchers built upon each other’s work.

Investment and Interest from Industry

• Advancement:
  • Growing interest and investment from tech giants and startups in AI technologies.
• Impact:
  • Led to the development of practical applications and commercial products, driving further research.

The 2000s: The Rise of Big Data and Deep Learning

Deep Learning Resurgence (2006)

• Milestone: Geoffrey Hinton and team introduced “deep belief networks.”
• Impact: Marked the resurgence of deep learning and laid the foundation for modern AI applications, especially in image and speech recognition.

AlexNet Wins ImageNet Competition (2012)

• Milestone: Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton’s deep neural network (AlexNet) won the ImageNet competition.
• Impact: Showcased the power of convolutional neural networks (CNNs) and triggered widespread adoption in computer vision tasks.

IBM Watson Wins Jeopardy! (2011)

• Milestone: IBM Watson defeated champions Ken Jennings and Brad Rutter on Jeopardy!.
• Impact: Demonstrated AI’s ability to process and understand natural language, leading to applications in healthcare, finance, and customer service.

MD Anderson sets Watson aside (2017-2018)

See Herper (n.d.)

Generative Adversarial Networks (GANs) (2014)

• Milestone: Ian Goodfellow introduced GANs, a model where two neural networks compete to generate realistic data.
• Impact: Revolutionized image generation and unsupervised learning, powering innovations like deepfakes and AI-generated art.

AlphaGo Defeats World Champion (2016)

• Milestone: Google DeepMind’s AlphaGo defeated Go champion Lee Sedol.
• Impact: Showcased the capability of reinforcement learning and deep neural networks in mastering complex strategic games.

Transformer Architecture (2017)

• Milestone: Vaswani et al. introduced the Transformer model, revolutionizing natural language processing.
• Impact: Laid the groundwork for state-of-the-art NLP models like BERT and GPT, transforming language understanding and generation.

AlphaFold Solves Protein Folding (2020)

• Milestone: DeepMind’s AlphaFold achieved breakthrough accuracy in predicting protein structures.
• Impact: Solved a 50-year-old challenge in biology, opening new doors in drug discovery and molecular biology.

GPT-4 and Large Language Models (2023)

• Milestone: OpenAI’s GPT-4 showcased the potential of large-scale language models for complex, nuanced language understanding.
• Impact: Accelerated the development of AI-driven content creation and enhanced human-computer interaction.

Impact

Nobel prize in chemistry

Nobel prize in physics

Future Directions

• Artificial General Intelligence (AGI) research
• Quantum computing and AI
• Neuromorphic computing
• Human-AI collaboration

For deeper dive into the history, see (Norman 2024).

Challenges and Opportunities

• Ethical AI development
• AI governance and regulation
• Addressing AI bias and fairness
• Balancing innovation with responsible development

References

Asimov, Isaac. 1950. I, Robot. Bantam hardcover ed. (2). New York: Bantam Books.

Herper, Matthew. n.d. “MD Anderson Benches IBM Watson In Setback For Artificial Intelligence In Medicine.” Forbes. Accessed October 2, 2024. https://www.forbes.com/sites/matthewherper/2017/02/19/md-anderson-benches-ibm-watson-in-setback-for-artificial-intelligence-in-medicine/.

Norman, Jeremy. 2024. “History of Information.” https://www.historyofinformation.com/?cat=71.

Turing, Alan. 1936. “On Computable Numbers, with an Application to the Entscheidungsproblem.” https://www.abelard.org/turpap2/tp2-ie.asp.

———. 1950. “Computing Machinery and Intelligence.” Mind 59 (October): 433–60. https://doi.org/10.1093/mind/lix.236.433.