Learning objectives
Objective of the course is to provide the student with the ability to understand and apply the basic rules of machine learning and, in particular:
- to apply the most common statistical tests in classification among different categories
- to synthesize the structure of the optimal classifier and analyze its error performance
- to apply the most common feature extraction methods from input data
- to apply the most common statistical estimators in machine learning
- to apply the most common clustering algorithms in unsupervised learning
The abilities in applying the above-mentioned knowledge are in particular in the:
- design and performance analysis of classifiers in machine learning
- selection of the most appropriate features to discriminate input categories
- selection of the most appropriate clustering algorithms in the design of unsupervised classifiers
Prerequisites
Entry-level courses in linear algebra and probability theory, such as those normally offered in the corresponding 3-year Laurea course, are necessary pre-requisites for this course.
Course unit content
PART 1: Fundamentals (Bononi)
(follows ref [1] Ch 1-9)
_____________________________________________
1. Introduction:
- problem statement and definitions
- Examples of machine learning problems
2. Basic probability refresher:
- Bayes formula
- conditional density functions
3. Classical Decision Rules
- binary Bayes rule
- M-ary Bayes rule
- receiver operating curve (ROC) and its properties
- Glossary of equivalent terms in Radar detecton theory, hypothesis testing and
machine learning
4. Linear Algebra refresher
- Unitary and Hermitian matrices
- spectral decomposition (SD)
- covariance matrices and diagonalization
5. Feature extraction
- sufficient statistics
- feature extraction based on eigenvector analysis
6. Quadratic and linear classifiers
- discriminant functions
- classification with Gaussian vectors
- Bounds on classifiers error probability
7. parameter estimation
- maximum likelihood and properties
- bayes estimation: MMSE and MAP
- Bounds on MS error
PART 2: Advanced topics (Cagnoni)
________________________________________________________
8. Nonparametric estimation
- Parzen density estimation
- k-Nearest-Neighbor algorithm
9. Linear Discriminant Analysis
- Fisher linear classifier
- Support Vector Machines
10. Classifier evaluation:
- Generalization and overfitting (Training/validation/test sets)
- Performance indices, representations curve, confusion matrices
- Classification risk: are all errors equally relevant ?
11. Unsupervised classification and clustering
- K-means and Isodata algorithms
- Self-Organizing Maps
- Learning Vector Quantization
- Kohonen networks
------------------------------------------------------------
Full programme
- - -
Bibliography
[1] C. W. Therrien, "Decision, estimation and classification" Wiley, 1989
[2] C. M. Bishop "Pattern Recognition and Machine Learning", Springer, 2006.
[3] R O Duda, P, E. Hart, D. G. Stork, "Pattern classification", 2nd Ed., Wiley, 2001
Teaching methods
Classroom teaching, 42 hours.
In-class problem solving, 6 hours.
Homework regularly assigned.
Assessment methods and criteria
Part 1, Bononi: Oral only, to be scheduled on an individual basis. When ready, please contact the instructor by email at alberto.bononi[AT]unipr.it and by specifying the requested date. The exam consists of solving some exercises and explaining theoretical details connected with them, for a total time of about 1 hour. You can bring your summary of important formulas in an A4 sheet to consult if you so wish.
Part 2, Cagnoni: A practical project will be assigned, whose results will presented and discussed by the student both as a written report and as an oral presentation.
Other information
Office Hours
Bononi: Monday 11:30-13:30 (Scientific Complex, Building 2, floor 2, Room 2/19T).
Cagnoni: by appointment (Scientific Complex, Building 1, floor 2, email cagnoni[AT]ce.unipr.it).
