**Code:**DS5005 |

**Category:**PMT |

**Credits:**3-0-0-3

# Topics

Introduction: Motivation and examples (3 hours)

Basics: Rd, vectors, matrices, norm, sequences & convergence, functions in one and several variables, Taylor series, derivatives, gradient, sub-gradient, Hessian, properties of symmetric operators, contours, affine functions, hyperplanes, convex functions, minima: local and global, subspaces, affine spaces, half-spaces, convex sets (9 hours)

Unconstrained Optimisation: gradient descent, line search, rates for various classes of convex functions, steepest descent, Newtonâ€™s method, conjugate gradient method, quasi-Newton method, linear least-squares regression: rates (9 hours)

Constrained Optimisation: linear and convex constraint sets, linear programming, simplex, interior-point method, duality theory: primal/dual programs, weak, strong duality, KKT conditions (12 hours)

Stochastic Optimisation: stochastic gradient descent, step-size conditions, Keifer-Wolwowitz method, simultaneous perturbation stochastic approximation (SPSA) method, smoothed functional method (9 hours)

# Learning Outcomes

As a result of this course, the student should be able to

- Pose a given optimisation problem by identifying the objective and constraints.
- Draw level sets and graphs of functions, identify constraint regions described by set of function.
- Choose stepsize and identify rates of convergence of optimisation algorithms.
- Use stochastic optimisation techniques to derive data driven algorithms .

# Learning Objectives

To

- Look at the regions and functions in `dâ€™ dimensions.
- Build algorithms that find minima using first and second order information.
- Look at constraint optimisation and duality theory with linear programming as a special case.
- Introduce basics of stochastic optimisation

# Text Books

- Introduction to Optimization: Edwin K. P. Chong and Stanislaw H. Zak, WileyInterscience Series in Discrete Mathematics and Optimization (ISBN-13: 9780471089490, ISBN-10: 0471089494).

# References

- Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004. (ISBN-13: 978-0521833783, ISBN-10: 0521833787)
- Fletcher, Roger. Practical methods of optimization. John Wiley & Sons, 2013. (ISBN:978-0-471-49463-8)

# Past Offerings

(Note: Past offerings could be under a different course number.)- Offered in Aug-Dec, 2022 by Sahely Bhadra
- Offered in Jul-Dec, 2021 by Sahely
- Offered in Jul-Dec, 2020 by Chandra Shekar

# Course Metadata

Item | Details |
---|---|

Course Title | Optimisation |

Course Code | DS5005 |

Course Credits | 3-0-0-3 |

Course Category | PMT |

Approved on | Senate of IIT Palakkad |

Course pre-revision code | CS5011 |