This course is about problem solving techniques. The problems in which we are interested in this course call for optimizing a certain quantity under certain conditions, and involve discrete rather than continuous quantities.
Many of the problems we face today require optimizing certain quantities. Scheduling of programs to be executed by computers, code optimization by a compiler, minimization of chip area for VLSI chips etc. are examples of this kind of problem.
Because of the practical importance of optimization, it has been studied extensively and the research still continues today. The solution techniques for optimization problems we have today (and also currently being investigated) fall into the following two classes: general methods and problem specific methods. General methods can be applied to a broad class of optimization problems and do not exploit idiosyncrasies of a specific problem on hand. Linear programming and its derivatives are examples of this type. Problem-specific methods, on the other hand, take advantage of structures that exist in the problem being solved. By their very nature, they can not be applied to other types of problem. They are nevertheless worth studying because one often finds helpful hints in them and can develop analogous solution methods.
This course concentrates on general methods. Problem specific approaches will be discussed elsewhere.
The first part of this course deals with deterministic methods such as linear programming, cutting plane methods, branch-and-bound, and dynamic programming. These are powerful techniques and quite useful in practice.
When a problem size becomes large, however, those techniques require a large amount of time and the problem becomes practically unsolvable (if it takes a year to solve a problem you might call it unsolvable for all practical purposes). Thus recently stochastic approaches have begun to draw attention. The second part of this course looks at some of the methods of this type. We shall study simulated annealing, genetic algorithms and distributed computing by neural network.
Another way to avoid spending a prohibitive amount of time solving a problem is to settle for approximate solutions rather than exact ones. In fact, the stochastic methods also produce an approximate solution if insufficient amount of time is spent on computation. There are also numerous deterministic approximation methods. Some of them may be covered in this course time permitting.
Toics to be Covered:
Linear Programming, Integer Programming including Greedy Methods, Branch-and-Bound Approach, Cutting Plane Methods, and Dynamic Programming, Simulated Annealing, Genetic Algorithms and Neural Net Approaches.
Other possible topics include optimization of nonlinear continuous functions (non-linear programming), matroids, and network flow problems.
Prerequisites: All CS undergraduate core subjects plus CS 600 or equivalent. Knowledge of Linear Algebra is also essential.
Textbook: No textbook. Notes will be distributed.
References:
1. Discrete Optimization in General
(a) C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization - algorithms and complexity, Prentice-Hall, 1982. QA402.5.P37
(b) R. G. Parker and R. L. Rardin, Discrete Optimization, Academic Press, 1988. QA402.5.P39
(c) F. S. Hillier and G. J. Lieberman, Operations Research, Holden-Day, 1974. T57.6.H54 (This book is very old. But it is a good introductory book on optimization.)
(d) G. L. Nemhauser and L. A. Wolsey, Integer and Combinatorial Optimization, Wiley, 1988. QA402.5.N453 (This is for advanced students and researchers.)
2. Linear Programming
In addition to chapters in the books of 1. above (a) V. Chvatal, Linear Programming, Freeman and Company, 1983.
(b) H. Karloff, Linear Programming, Birkhauser, 1991.
3. Genetic Algorithm
(a) D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, 1989. QA402.5.G635 (This is a very good introductory book.)
4. Simulated Annealing
(a) E. H. L. Aarts and J. Korst, Simulated Annealing and Boltzmann Machines - a stochastic approach to combinatorial optimization and neural computing , Wiley, 1989. QA402.5.A22 (This is for technically advanced people. But reading the first two chapters skipping proofs gives you a good overview of the subject.)
5. Neural Nets
Chapter 8 of Aarts and Korst gives the essence of neural net very nicely and Chapter 9 presents examples of combinatorial optimization problem solved using neural nets. (a) J. M. Zurada, Introduction to Artificial Neural Systems, West Publishing Company, 1992.