Sample Syllabus for a Semester Course on
Nonlinear Programming

using Optimization in Operations Research
by Ronald L. Rardin, Prentice Hall, 1998

Audience

This course is intended as an advanced undergraduate or beginning graduate course in nonlinear programming for students in engineering, management and mathematics.

Background

This course assumes elementary background in multivariate differential calculus and in linear algebra, plus familiarity with vector/matrix notation and arithmetic. Most students will also have had an introductory course in deterministic operations research that includes at least formulation of mathematical programs plus the simplex method and duality for linear programming. Still, self study of Chapter 2, Sections 5.1-5.3 and Sections 7.3-7.5 can serve as an adequate substitute.

Topics

Schedule (less one week of examinations)

Week Topics Reading
1 Class Organization
Nature and Diversity of Nonlinear Programs
Section 2.4
2 Improving Search Paradigm
Improving and Feasible Directions		 Sections 3.1
Sections 3.2-3.3
3 Local and Global Optima
Initial Feasible Solutions 		Section 3.4
Section 3.5
4 Formulation on Unconstrained NLPs
Golden Section Search		 Section 13.1
Section 13.2
5 Bracketing and Quadratic Fit Search
First and Second Order Conditions for Local Optima		 Section 13.2
Section 13.3
6 Convex and Concave Functions
Gradient Search		 Section 13.4
Section 13.5
7 Newton's Method
Convergence of NLP Algorithms
Quasi-Newton Algorithms		 Section 13.6
Supplement 1
Section 13.7
8 Conjugate Gradient Methods
Unconstrained Optimization without Derivatives		 Supplement 2
Section 13.8
9 Formulation of Constrained Nonlinear Programs
Convex, Separable, Quadratic and Geometric Programming Cases		 Section 14.1
Section 14.2
10 Lagrange Multiplier Techniques
Karush-Kuhn-Tucker Optimality Conditions		 Section 14.3
Section 14.4
11 Penalty and Barrier Methods Section 14.5
12 Review of Simplex for LP
Reduced Gradient Methods		 Sections 5.1-5.3
Section 14.5
13 Quadratic Programming
Sequential Quadratic Programming Methods		 Section 14.7
Supplement 3
14 Separable Programming
Posynomial Geometric Programming		 Section 14.8
Section 14.9

Supplements

Although the text provides foundations and intuitions for nearly all topics of this course, instructors will need to add rigor where the level of the course requires formal proofs. Also, the following supplemental notes are suggested:
  1. Convergence of NLP Algorithms: Notions of NLP algorithm convergence and proofs based on Taylor series.
  2. Conjugate Gradient Methods: An introduction at the same level as the text to conjugate gradient methods for unconstrained nonlinear programming.
  3. Sequential Quadratic Programming: An introduction at the same level as the text to sequential quadratic programming methods for constrained nonlinear programming.

Computer Support

The course should be conducted in part using standard class optimization software such as GAMS, AMPL, or LINGO. It is strongly recommended that students use such software to solve all assigned formulations that have numbers.

Still, the combination of simple hand exercises and full "black box" optimization is often not sufficient for students to master NLP algorithm strategies. Better learning can result from assigning students to program and run their own crude versions of the more straight-forward nonlinear methods.

Back to Optimization in Operations Research mainpage