Prerequisite: Familiarity with Algorithms, Probability, Linear Algebra, Programming

Course Content

  1. Introduction to the course, revision of linear algebra and probability (3 hours)

  2. Regression: linear regression, ridge regression (3 hours)

  3. Classification: (9 hours)
    • Linear discriminant analysis, logistic regression, perceptrons,
    • support vector machines, Bayes classifier, decision tree.
    • Nonparametric methods: k-nearest neighbours, Parzen window.
  4. Principal component analysis, Canonical correlation analysis (3 hours)

  5. Evaluation and Model Selection: ROC Curves, Evaluation Measures, Cross validation, Significance tests (3 hours)

  6. Ensemble methods: boosting, bagging, random forests (3 hours)

  7. Clustering: (9 hours)
    • k-means, hierarchical, density based clustering
    • Gaussian mixture model
  8. Sequential Learning : hidden Markov model (6 hours)

  9. Neural network : feedforward NN (3 hours)

Learning Outcomes

  • State definitions, theorems/results, algorithms related to key concepts
  • Apply standard techniques to solve known problems
  • Given a task, derive a learning model by defining appropriate loss function, regulariser, optimization problem and stating the best possible solution.
  • Analyse and compare models and algorithms with respect to their complexity, performance and applicability
  • Develop models/algorithms with small modifications of existing standard techniques for a modification of known task

Text Books

  1. Richard Duda, Peter Hart, David Stork, Pattern Classification, 2nd Ed, John Wiley & Sons, 2001. ISBN 9788126511167
  2. Christopher Bishop. ​Pattern Recognition and Machine Learning​. ISBN 0387310738.
  3. Trevor Hastie, Robert Tibshirani, Jerome Friedman. Elements of Statistical Learning. ISBN 0387952845.

References

  1. Tom Mitchell. Machine Learning. McGraw-Hill. ISBN 0070428077.
  2. Shai Shalev-Shwartz, and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014. ISBN 978-1-107-05713-5.