Learning objectives
Course Objectives
-----------------
The objective of the course is to provide the student with the ability to
understand and apply the basic rules of machine learning and, in
particular:
- to apply the most common statistical tests in classification among
different categories
- to synthesize the structure of the optimal classifier and analyze its error
performance
- to apply the most common feature extraction methods from input data
- to apply the most common statistical estimators in machine learning
- to apply the most common clustering algorithms in unsupervised
learning
The abilities in applying the above-mentioned knowledge are in particular
in the:
- design and performance analysis of classifiers in machine learning
- selection of the most appropriate features to discriminate input
categories
- selection of the most appropriate clustering algorithms in the design of
unsupervised classifiers
Prerequisites
Entry-level courses in linear algebra and probability theory, such as those
normally offered in the corresponding 3-year Laurea course, are necessary
pre-requisites for this course.
Course unit content
PART 1: Fundamentals (Bononi)
(follows ref [1] Ch 1-9)
_____________________________________________
1. Introduction:
- problem statement and definitions
- Examples of machine learning problems
2. Basic probability refresher:
- Bayes formula
- conditional density functions
3. Classical Decision Rules
- binary Bayes rule
- M-ary Bayes rule
- receiver operating curve (ROC) and its properties
- Glossary of equivalent terms in Radar detecton theory, hypothesis
testing and machine learning
4. Linear Algebra refresher
- Unitary and Hermitian matrices
- spectral decomposition (SD)
- covariance matrices and diagonalization
5. Feature extraction
- sufficient statistics
- feature extraction based on eigenvector analysis
6. Quadratic and linear classifiers
- discriminant functions
- classification with Gaussian vectors
- Bounds on classifiers error probability
7. parameter estimation
- maximum likelihood and properties
- bayes estimation: MMSE and MAP
- Bounds on MS error
PART 2: Advanced topics and applications (Cagnoni)
________________________________________________________
8. Nonparametric estimation
- Parzen density estimation
- k-Nearest-Neighbor algorithm
9. Linear Discriminant Analysis
- Fisher linear classifier
- Support Vector Machines
10. Classifier evaluation:
- generalization and overfitting (Training/validation/test sets)
- performance indices, representations curve, confusion matrices
- Classification risk: are all errors equally relevant ?
11. Unsupervised classification and clustering
- K-means and Isodata algorithms
- Self-Organizing Maps
- Learning Vector Quantization
- Kohonen networks
Full programme
- - -
Bibliography
[1] C. W. Therrien, "Decision, estimation and classification" Wiley, 1989
[2] C. M. Bishop "Pattern Recognition and Machine Learning", Springer,
2006.
[3] R O Duda, P, E. Hart, D. G. Stork, "Pattern classification", 2nd Ed.,
Wiley, 2001
Teaching methods
Classroom teaching, 42 hours.
In-class problem solving, 6 hours.
Homework regularly assigned.
Assessment methods and criteria
Part 1, Bononi: Oral only, to be scheduled on an individual basis. When
ready, please contact the instructor by email at alberto.bononi[AT]unipr.
it and by specifying the requested date. The exam consists of solving
some exercises and explaining theoretical details connected with them,
for a total time of about 1 hour. You can bring your summary of important
formulas in an A4 sheet to consult if you so wish.
Part 2, Cagnoni: A practical project will be assigned, whose results will be
presented and discussed by the student both as a written report and as
an oral presentation.
Other information
Office Hours
Bononi: Monday 11:30-13:30 (Scientific Complex, Building 2, floor 2,
Room 2/19T).
Cagnoni: by appointment (Scientific Complex, Building 1, floor 2, email
cagnoni[AT]ce.unipr.it).
