Course

Unit I

Introduction to optimization: classical optimization, Optimality criteria – Necessary and sufficient conditions for existence of extreme point.

Direct search methods: unidirectional search, evolutionary search method, simplex search method, Introduction,Conditions for local minimization. One dimensional Search methods: Golden search method, Fibonacci method, Newton’s Method, Secant Method, Remarks on Line Search Sections.  Hook-Jeeves pattern search method.

Unit II

Gradient-based methods– introduction, the method of steepest descent, analysis of Gradient Methods, Convergence, Convergence Rate.  Analysis of Newton’s Method, Levenberg-Marquardt Modification, Newton’s Method for Nonlinear Least-Squares.

Conjugate direction method, Introduction, The Conjugate Direction Algorithm, The Conjugate Gradient Algorithm for Non-Quadratic Quasi Newton method.

Unit III

Nonlinear Equality Constrained Optimization– Introduction, Problems with equality constraints Problem Formulation, Tangent and Normal Spaces, Lagrange Condition

Nonlinear Inequality Constrained Optimization -Introduction – Problems with inequality constraints: Kuhn-Tucker conditions.

Lab Practice Problems: Single and multivariable optimizations. Implementation of iterative methods.

Case studies

Course Objectives

• To understand the concept of search space and optimality for solutions of engineering problems.
• To understand some computation techniques for optimizing single variable functions.
• To understand various computational techniques for optimizing severable variable functions.

Course Outcomes

CO1: Understand different types of Optimization Techniques in engineering problems. Learn Optimization methods such

as Bracketing methods, Region elimination methods, Point estimation methods.

CO2: Understand Optimizations Techniques in single variable functions.

CO3: Understand the optimality criteria for the multivariable optimizations.

CO4: Understand Optimizations Techniques in multi variable functions.

CO5: Understand constrained optimization techniques and Kuhn-Tucker conditions.

CO-PO Mapping

Evaluation Pattern: 70:30

Textbook(s)

Edwin K.P. Chong, Stanislaw H. Zak, “An introduction to Optimization”, 2^nd edition, Wiley, 2013.

Reference(s)

S.S. Rao, “Optimization Theory and Applications”, Second Edition, New Age International (P) Limited Publishers, 1995.

Kalyanmoy Deb, “Optimization for Engineering Design: Algorithms and Examples, Prentice Hall, 2002.

Lab Experiments:

1. Identifying definiteness of matrices using eigenvalues and use of Hessian matrix to identify concavity of the surfaces
2.     (revision from Calculus and Linear Algebra)
3. Implementation of Golden Section Search, Fibonacci search for single variable optimization problems
4. Evaluation of ordinary and partial derivatives numerically (in excel/MATLAB)
5. Implementation of Secant method and Newton’s method for single variable optimization problems
6. Implementation of evolutionary search method for multivariable optimization problems
7. Implementation of Simplex search method for multivariable optimization problems
8. Implementation of Hooke-Jeeve’s Pattern Search method for multivariable optimization problems
9. Implementation of Newton’s method for solving system of non-linear equations
10. Implementation of Newton’s method for solving multivariable optimization problems
11. Identifying whether a constrained optimization problem is convex or not and solutions using ‘cvx’