CS 600 Algorithms and Data Structures

Notes on Linear Programming

Simplex Method for Linear Programming Problem

The simplex method is a systematic way to solve an LP problem. It finds an optimal solution by visiting basic feasible solutions one by one starting at some initial basic feasible solution until the value of the objective function can not be improved any more.

Given the following LP problem,

Maximize  $\Sigma_{j=1}^{n} c_{j} x_{j}$ --- (1)
Subject to $\Sigma_{j=1}^{n} a_{ij} x_{j} \leq b_{i}$ ,   i = 1,2,...,m --- (2)
                 $x_{j} \geq 0$ ,   j = 1,2, ..., n --- (3).
the simplex method proceeds as follows:

1. Introduce a new set of variables, $x_{n+i} \geq 0$, i = 1,2, ..., m, one for each inequality of (2), and convert the inequalities of (2) to equalities. These newly introduced variables are called slack variables.
Thus (2) now becomes

                $\Sigma_{j=1}^{n} a_{ij} x_{j} + x_{n+i} = b_{i}$ ,   i = 1,2,...,m --- (4), and
(3) becomes

                $x_{j} \geq 0$ ,   j = 1,2, ..., n+m --- (5).

2. Find an initial basic feasible solution.
For this, set x_j = 0 for j = 1,2, ..., n, and find the value of the remaining variables from (4). The variables x_n+1, ..., x_n+m are called the basic variables, and x₁, ..., x_n are called the nonbasic variables. This selection of basic and nonbasic variables does not necessarily produce a feasible solution. But here we assume that is gives a feasible solution. The other case is discussed elsewhere(link???).

Then express the objective function in terms of the nonbasic variables.
Let us denote the objective function by Z.

3. Find a better basic feasible solution if one exists.

If the coefficient of the nonbasic variables in Z are all negative, then Z can not be improved by increasing their value, that is by moving to another basic feasible solution. Hence if that is the case, we stop and the current basic feasible solution is an optimal solution.

Selection of new basic variable: If some coefficients are positive, then select one of the variables with the largest coefficient. Let us denote it by x_p. Then x_p is going to be made a new basic variable to produce a new basic feasible solution. Since the number of basic variables remains unchanged throughout the process, one of the old basic variables must become nonbasic.

Selection of new nonbasic variable: The new nonbasic variable is the old basic variable whose value can be made 0 by the smallest value of x_p while keeping the other nonbasic variables 0 and the new set of basic variables nonnegative.

The process of finding a new basic feasible solution by these two operations is called a pivoting.

4. Repeat 3. until an optimal solution is found.


More formally the pivoting (Step 3. above) can be described as follows:

Let x_bi, i = 1,2, ..., m denote the current basic variables, and let x_nj, j = 1,2, ..., n be the current nonbasic variables. Also let Z represent the objective function.
Then each x_bi and Z can be expressed in terms of the nonbasic variables x_njas

$x_{bi} = r_{i} + \Sigma_{j=1}^{n} s_{ij} x_{nj}$, i = 1,2, ..., m
$Z = z_{0} + \Sigma_{j=1}^{n} c_{j} x_{nj}$
where r_j, c_j, s_ij and z₀ are constants.

If $c_{j} \leq 0$ for all j, then Z is optimum at the current basic feasible solution. Note that for the current basic feasible solution, x_nj = 0 for j = 1,2, ..., n and x_bi = r_i for i = 1,2, ..., m.

If c_j > 0 for some j, then x_nj with the largest c_j is selected to be the new basic variable (to break a tie, select arbitrary one).
For the new nonbasic variable, select, among the current basic variables, x_bi with the smallest -r_i/s_ij and $s_{ij} \leq 0$, where j in s_ijis the same j in the x_nj that has been selected to become the new basic variable.
Note that by this selection it is guaranteed that the new basic variables are nonnegative when the new nonbasic variables are set to 0.

Example of Simplex Method:

Consider the following LP problem:

Maximize   Z = x₁ + x₂
Subject to $-x_{1} + 2x_{2} \leq 6$
                $2x_{1} + x_{2} \leq 8$
                $x_{1}, x_{2} \geq 0$

Step 1. Introduce the slack variables x₃ and x₄ for the first and second inequalities, respectively. The inequalities are converted to the following equalities:

-x₁ + 2x₂ + x₃ = 6
2x₁ + x₂ + x₄ = 8


Step 2. Select x₃ and x₄ as the basic variables, and x₁ and x₂ as the nonbasic variables. Then

subject to
x₃ = 6 + x₁ - 2x₂
x₄ = 8 - 2x₁ - x₂ , and
x_i's are greater than or equal to 0 for i = 1, .., 4,
Z = x₁ + x₂
is to be maximized.

Since x₁ = x₂ = 0, x₃ = 6, x₄ = 8 is a feasible solution, it can be used as the initial basic feasible solution.

Step 3. Select x₁ as the new basic variable (since x₁ and x₂ have the same coefficient 1 in Z, x₂ could also be selected as the new basic variable). Then x₄ is selected for the new nonbasic variable since the coefficient of x₁is negative (-2) only in the equation for x₄.
Thus the new basic variables are x₁ and x₃, and the new nonbasic variables are x₂ and x₄.

We now have the second basic feasible solution, and Steps 2 and 3 are repeated for this new basic feasible solution.

Step 4. Representing the new basic variables and Z in terms of the new nonbasic variables we obtain:

x₃ = 10 - (5/2)x₂ - (1/2)x₄
x₁ = 4 - (1/2)x₂ - (1/2)x₄
Z = 4 + (1/2)x₂ -(1/2) x₄

Step 5. Select x₂ as the new basic variable since it is the only variable with a positive coefficient in Z. Then x₃ is selected to become the new nonbasic variable since 10/(5/2) is smaller than 4/(1/2).

Thus the new basic variables are x₁ and x₂, and the new nonbasic variables are x₃ and x₄.

We now have the third basic feasible solution, and Steps 2 and 3 are repeated for this new basic feasible solution.

Step 6. Representing the new basic variables and Z in terms of the new nonbasic variables we obtain:

x₂ = 4 - (1/5)x₄ - (2/5)x₃
x₁ = 2 - (2/5)x₄ + (1/5)x₃
Z = 6 - (3/5)x₄ - (1/5) x₃

Step 7. Since all the coefficients in Z are negative, Z can not be increased any further. Thus we have the optimum. The optimal solution is x₃ = x₄ = 0, x₁ = 2 and x₂ =4, and the optimum value is Z = 6.