An ontology of AI.
Gentle Introduction to Machine Learning
With a Focus on Supervised Learning
October 31, 2024
Supervised Learning: Classification
Supervised Learning: Regression
Supervised Learning: Regression
This problem could also be framed as a classification problem if we were to categorize BMI into categories (e.g., underweight, normal, overweight, obese).
Unsupervised Learning
Single Cell sequencing. Each point represents a single cell with the location in the plot based on gene expression.
The cell types are usually UNKNOWN when the data are generated.
Learning.
Prediction.
Model
- Convolutional Neural Network
- Weights and biases
- Activation functions
CAT
OR
DOG
Model
- Predictors/Features
- Regression coefficients
A map of machine learning approaches
Regression
Not all regression is linear….
Classification Example
The classic iris
Classification Example: Data Exploration and Visualization
Training and Testing
- Training Data: Used to teach the model. It consists of input-output pairs for a supervised learning task.
- Testing Data: Used to evaluate the model’s performance. It should be separate from the training data, but come from the same distribution or population.
- Validation Data: Used for tuning hyperparameters. Hyperparameters are settings that control the learning process (e.g., k in k-NN, learning rate in neural networks).
Classification Example
Classification Example
Classification Example
Regression Example
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-49.071 -11.047 -0.692 12.970 41.897
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.72808 3.68575 -0.198 0.844
x 2.05022 0.06336 32.356 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 18.29 on 98 degrees of freedom
Multiple R-squared: 0.9144, Adjusted R-squared: 0.9135
F-statistic: 1047 on 1 and 98 DF, p-value: < 2.2e-16Regression Example
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-98.143 -22.095 -1.385 25.940 83.795
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.4562 7.3715 -0.198 0.844
x 2.1004 0.1267 16.574 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 36.58 on 98 degrees of freedom
Multiple R-squared: 0.7371, Adjusted R-squared: 0.7344
F-statistic: 274.7 on 1 and 98 DF, p-value: < 2.2e-16Regression Example
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-171.749 -38.665 -2.423 45.395 146.641
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -2.5483 12.9001 -0.198 0.844
x 2.1758 0.2218 9.811 3.12e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 64.02 on 98 degrees of freedom
Multiple R-squared: 0.4955, Adjusted R-squared: 0.4904
F-statistic: 96.25 on 1 and 98 DF, p-value: 3.116e-16More Complex Regression Example
What do you think about the relationship between age and insurance charges?
More Complex Regression Example
Linear Regression
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-24.5356 -5.5236 -0.3462 6.4850 20.9487
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.36404 1.84287 -0.198 0.844
x 2.02511 0.03168 63.920 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 9.145 on 98 degrees of freedom
Multiple R-squared: 0.9766, Adjusted R-squared: 0.9763
F-statistic: 4086 on 1 and 98 DF, p-value: < 2.2e-16Algorithms for Supervised Learning
Anscombe’s Quartet. Each of these datasets has the same \(R^2\), slope, and p-value.
Algorithms for Supervised Learning
K-nearest neighbor (kNN) algorithm.
Algorithms for Supervised Learning
Classification and Regression Trees (CART).
Algorithms for Supervised Learning
Random Forests.
Algorithms for Supervised Learning
Deep learning.
Bias
Sources of bias in machine learning.
Bias
Mitigating bias in machine learning.
Applying Supervised Learning Algorithms
The mlr3 ecosystem in R.