The Simplex Method: Part 1

Canonical Forms - 6 Examples

The LP in Two Dimensions

Solving for $x_3, x_4$ in terms of $x_1$ and $x_2$, the constraints become \[x_1, x_2 \ge 0, \qquad x_1 + 2x_2 \le 3, \qquad x_1 + x_2 \le 2.\]

The 6 basic solutions are intersections of the lines, and the 4 basic feasible solutions are the intersections that lie in the feasible region.

The LP in Two Dimensions

The simplex method makes use of the edges in this picture to move from one feasible point to another.

What do these edges look like in algebra?

Pictures and Algebra

The edges go between bases that differ by one element.

Simplex Iterations

At each step of the simplex algorithm, we have a basis in mind (and an LP in canonical form with respect to that basis).

Each iteration, one element of the basis leaves and one nonbasis element enters, so that the number of basis elements stays the same.

For now, the element that enters can be any that has a positive coefficient in the objective, and the element that leaves is the one that passes the ratio test.

At the new basis, we need to put the LP into canonical form again with respect to the new basis.

We terminate when the coefficient vector has no positive entries, or when we find that the LP is unbounded (we'll discuss this later).

A Simplex Iteration

Start with a basic feasible solution

In pictures, we want to find an edge of the polytope and move along it to get to a new basic feasible solution with better objective. How do we do this?

We need to pick an entry where the objective vector has a positive coefficient (here either $3$ or $4$).

A Simplex Iteration

Let's pick the 4th one (last time we picked the third one)

Variable $x_4$ is entering. Which variable leaves?

A Simplex Iteration

Let's pick the 4th one (last time we picked the third one)

The ratio test says that we should compare the 4th column of $A$ to the right side of the equations $b$: \[ A_4 = \begin{pmatrix}2 \\ -1\end{pmatrix}\qquad b = \begin{pmatrix}1 \\ 1\end{pmatrix}. \] The ratios are $\frac{1}{2}$ and $\frac{1}{-1}$. The smallest nonnegative ratio is the first one so $x_1$ leaves.

A Simplex Iteration

Let's pick the 4th one (last time we picked the third one)

Since $x_4$ entered, and $x_1$ leaves, we should update the basis from $\{1,2\}$ to $\{1,2\} \setminus \{1\} \cup \{4\} = \{2,4\}$. Now, we need to put the system back in RREF.

A Simplex Iteration

Let's pick the 4th one (last time we picked the third one)

Since $x_4$ entered, and $x_1$ leaves, we should update the basis from $\{1,2\}$ to $\{1,2\} \setminus \{1\} \cup \{4\} = \{2,4\}$. Now, we need to put the system back in RREF.

A Simplex Iteration

Here is the system (in RREF with respect to $\{\color{orange}1,\color{blue}2\}$). We want to row reduce with respect to the basis $\{\color{blue}2,\color{red}4\}$.

\[\begin{pmatrix} \color{orange}1 & \color{blue}0 & -1 & \color{red}2 \\ \color{orange}0 & \color{blue}1 & 1 & \color{red}{-1}\end{pmatrix} x = \begin{pmatrix} 1 \\ 1\end{pmatrix}\]

We ultimately want the system to look like \[\begin{pmatrix} ? & \color{blue}1 & ? & \color{red}0 \\ ? & \color{blue}0 & ? & \color{red}1\end{pmatrix} x = \begin{pmatrix} ? \\ ?\end{pmatrix}\]

We can fix the blue column by swapping the two rows.

A Simplex Iteration

Here is the system with the rows swapped

\[\begin{pmatrix} \color{orange}0 & \color{blue}1 & 1 & \color{red}{-1} \\ \color{orange}1 & \color{blue}0 & -1 & \color{red}2\end{pmatrix} x = \begin{pmatrix} 1 \\ 1\end{pmatrix}\]

This makes the second column already correct. Now, we just need to fix the 4th column to put it in RREF.

A Simplex Iteration

Here is the system we want to row reduce.

\[\begin{pmatrix} \color{orange}0 & \color{blue}1 & 1 & \color{red}{-1} \\ \color{orange}1 & \color{blue}0 & -1 & \color{red}2\end{pmatrix} x = \begin{pmatrix} 1 \\ 1\end{pmatrix}\]

\[\downarrow_{\rho_2 = \rho_2/2}\]

\[\begin{pmatrix} \color{orange}0 & \color{blue}1 & 1 & \color{red}{-1} \\ \color{orange}{\frac{1}{2}} & \color{blue}0 & -\frac{1}{2} & \color{red}1\end{pmatrix} x = \begin{pmatrix} 1 \\ \frac{1}{2}\end{pmatrix}\]

\[\downarrow_{\rho_1 = \rho_1 + \rho_2}\]

\[\begin{pmatrix} \color{orange}{\frac{1}{2}} & \color{blue}1 & \frac{1}{2} & \color{red}0 \\ \color{orange}{\frac{1}{2}} & \color{blue}0 & -\frac{1}{2} & \color{red}1\end{pmatrix} x = \begin{pmatrix} \frac{3}{2} \\ \frac{1}{2}\end{pmatrix}\]

A Simplex Iteration

We have the equivalent LP

This is still not in RREF; to put it in RREF, subtract 4 times the second row from the objective.

A Simplex Iteration

We have the equivalent LP

This is still not in RREF; to put it in RREF, subtract 4 times the second row from the objective.

A Simplex Iteration

We have the equivalent LP

This is now in canonical form with respect to $\{2,4\}$.

A Simplex Iteration

We have the equivalent LP

What should our next entering variable be?

It has to be variable 3, since it is the only one with a positive coefficient in the objective.

A Simplex Iteration

Variable 3 enters, what variable leaves? The ratio test asks us to look at \[ A_3 = \begin{pmatrix}\frac{1}{2}\\-\frac{1}{2}\end{pmatrix} \qquad b = \begin{pmatrix}\frac{3}{2} \\ \frac{1}{2} \end{pmatrix} \qquad b / A_3 = \begin{pmatrix}\tfrac{\frac{3}{2}}{\frac{1}{2}} \\ \tfrac{\frac{1}{2}}{-\frac{1}{2}}\end{pmatrix}. \]

A Simplex Iteration

In the vector $ b / A_3 = \begin{pmatrix}\tfrac{\frac{3}{2}}{\frac{1}{2}} \\ \tfrac{\frac{1}{2}}{-\frac{1}{2}}\end{pmatrix}$, the only positive entry is the first one, so the first basis element (2) leaves (the one with the 1 in the first row).

The new basis is $\{2,4\} \setminus \{2\} \cup \{3\} = \{3,4\}$.

A Simplex Iteration

We now have to put the following into canonical form with respect to $\{3,4\}$.

First row reduce the linear system.

A Simplex Iteration

Column 4 is already correct; we now need to perform row operations to fix the 3rd column.

A Simplex Iteration

\[ \begin{pmatrix} {\frac{1}{2}} & \color{orange}1 & \color{red}{\frac{1}{2}} & \color{blue}0 \\ {\frac{1}{2}} & \color{orange}0 & \color{red}{-\frac{1}{2}} & \color{blue}1\end{pmatrix} x = \begin{pmatrix} \frac{3}{2} \\ \frac{1}{2}\end{pmatrix} \] \[ \downarrow_{\rho_1 = 2\rho_1} \] \[ \begin{pmatrix} 1 & \color{orange}2 & \color{red}{1} & \color{blue}0 \\ {\frac{1}{2}} & \color{orange}0 & \color{red}{-\frac{1}{2}} & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 3 \\\frac{1}{2}\end{pmatrix} \] \[ \downarrow_{\rho_2 = \rho_2 + \frac{1}{2}\rho_1} \] \[ \begin{pmatrix} 1 & \color{orange}2 & \color{red}{1} & \color{blue}0 \\ 1 & \color{orange}1 & \color{red}0 & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 3 \\ 2\end{pmatrix} \]

A Simplex Iteration

Now. we have the equivalent LP

We again eliminate coefficient 3 in the objective by subtracting a multiple of row 1 from the objective.

A Simplex Iteration

Now. we have the equivalent LP

We again eliminate coefficient 3 in the objective by subtracting a multiple of row 2 from the objective.

A Simplex Iteration

Now. we have canonical form LP

The objective vector has only nonpositive entries, so this is optimal.

A Simplex Iteration

Why does the ratio test work for determining a leaving variable? Why does the objective function go up in each iteration?

For an LP in canonical form with respect to a basis $B = \{\beta_1, \dots, \beta_m\}$, the basic feasible solution $x_b$ is a vertex. For a given $k \not \in B$, an edge direction is defined to be a vector $v$ so that \[ Av = 0 \qquad c^{\intercal}v > 0 \qquad v_{k} = 1 \qquad \forall j \not \in B \cup \{k\}, v_j = 0. \]

If $v$ is an edge direction, then \[ x^{(t)} = x_b + tv \] satisfies the linear constraints since $A(x_b + tv) = Ax_b = b$. Also, $c^{\intercal} x^{(t)} = c^{\intercal}x_b + t c^{\intercal}v$ is increasing in $t$.

A Simplex Iteration

\[ x^{(t)} = x_b + tv \]

$x^{(t)}$ is therefore feasible as long as it is nonnegative; it is nonnegative as long as $(x_b)_i + tv_i \ge 0$ for each $i$.

Note that if $i$ is not basic and also not the entering variable $k$, then $x_i = v_i = 0$ by definition of the edge direction and the basic feasible solution, so these entries are nonnegative.

Also, if $v_i \ge 0$, then $(x_b)_i + tv_i \ge 0$ for $t \ge 0$, so these also aren't interesting. In particular, the $k$th entry is always $t \ge 0$.

The only relevant constraints are that whenever $i$ is in the basis, and $v_i < 0$, $t \le -\frac{x_i}{v_i}$.

A Simplex Iteration

What are the entries of $x^{(t)}_i$ and $v_i$ when $i = \beta_{\ell}$ is in the basis?

We have seen that $x_{\beta_{\ell}} = b_{\ell}$.

It also happens that the edge direction satisfies $v_{\beta_{\ell}} = - A_{\ell k}$ for each $\ell$, which can be checked by solving a linear system of equations.

So, the $t$ that works is $t = \min_{\ell : A_{\ell k} > 0} \frac{b_{\ell}}{A_{\ell k}}$, and the leaving variable (that goes to $x$ in $x$ that goes to 0) can be any $\ell$ so that this minimum is achieved.

Simplex Explorer

We really just need to keep track of the canonical form in each iteration.

Another Simplex Iteration

Here is a bigger LP ($3$ constraints, $5$ variables), in canonical form with respect to the basis $\{1,3,5\}$.

What is the basic feasible solution?

The basic feasible solution is $x = (2, 0, 6, 0, 4)$. We can enter either $x_2$ or $x_4$ (both have positive objective coefficients).

Another Simplex Iteration

Let's enter $x_4$. Which variable leaves?

\[ A_4 = \begin{pmatrix}1 \\ 2 \\ -1\end{pmatrix} \qquad b = \begin{pmatrix}2 \\ 6 \\ 4\end{pmatrix} \qquad b / A_4 = \begin{pmatrix}\frac{2}{1}\\ \frac{6}{2} \\ \frac{4}{-1}\end{pmatrix}. \] The ratio test compares the $4$th column $A_4$ with $b$: The smallest is $2$, in the first row, so $x_1$ leaves.

Another Simplex Iteration

$x_4$ entered and $x_1$ left. $\{1,3,5\} \setminus \{1\} \cup \{4\} = \{3,4,5\}$. Put the system back in RREF with respect to $\{3,4,5\}$.

\[\begin{pmatrix} \color{orange}1 & 0 & \color{blue}0 & \color{red}1 & \color{blue}0 \\ \color{orange}0 & -1 & \color{blue}1 & \color{red}2 & \color{blue}0 \\ \color{orange}0 & -1 & \color{blue}0 & \color{red}{-1} & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 2 \\ 6 \\ 4\end{pmatrix}\]

We would like the final matrix to look like \[\begin{pmatrix} ? & ? & \color{blue}1 & \color{red}0 & \color{blue}0 \\ ? & ? & \color{blue}0 & \color{red}1 & \color{blue}0 \\ ? & ? & \color{blue}0 & \color{red}0 & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 6 \\ 2 \\ 4\end{pmatrix}\]

If we swap rows 1 and 2, then the blue are already done (the swapping is because the leaving variable was the first variable in the basis, and the entering variable will be the second).

Another Simplex Iteration

Now, we have an easier time row reducing the following:

\[\begin{pmatrix} \color{orange}0 & -1 & \color{blue}1 & \color{red}2 & \color{blue}0 \\ \color{orange}1 & 0 & \color{blue}0 & \color{red}1 & \color{blue}0 \\ \color{orange}0 & -1 & \color{blue}0 & \color{red}{-1} & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 6 \\ 2 \\ 4\end{pmatrix}\downarrow_{\rho_1 = \rho_1 - 2\rho_2}\]

\[\begin{pmatrix} \color{orange}{-2} & -1 & \color{blue}1 & \color{red}0 & \color{blue}0 \\ \color{orange}1 & 0 & \color{blue}0 & \color{red}1 & \color{blue}0 \\ \color{orange}0 & -1 & \color{blue}0 & \color{red}{-1} & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 2 \\ 2 \\ 4\end{pmatrix}\downarrow_{\rho_3 = \rho_3 + \rho_2}\]

\[\begin{pmatrix} \color{orange}{-2} & -1 & \color{blue}1 & \color{red}0 & \color{blue}0 \\ \color{orange}1 & 0 & \color{blue}0 & \color{red}1 & \color{blue}0 \\ \color{orange}1 & -1 & \color{blue}0 & \color{red}0 & \color{blue}1\end{pmatrix} x = \begin{pmatrix} 2 \\ 2 \\ 6\end{pmatrix}\]

Another Simplex Iteration

We have the equivalent LP (the constraints are in RREF for $\{3,4,5\}$):

The objective still has a nonzero coefficient on the basic variable $x_4$. Subtract $3$ times row $2$ from the objective to price it out.

Another Simplex Iteration

We have the equivalent LP

This gives canonical form with respect to $\{3,4,5\}$.

Another Simplex Iteration

Canonical form with respect to $\{3,4,5\}$, with basic feasible solution $x = (0,0,2,2,6)$ and objective value $6$:

The only positive objective coefficient is on $x_2$, so $x_2$ enters next.

Another Simplex Iteration

Entering $x_2$: what happens when we perform the ratio test?

Column $A_2$ (red) has no positive entry, so the ratio test has no limiting row: we can increase $x_2 = t$ forever, increasing the objective while keeping things feasible.

The LP is unbounded.

Another Simplex Iteration

In summary, we can keep performing these simplex iterations, each time picking an entering variable with positive objective coefficient and using the ratio test to determine the leaving variable.

Two ways to terminate:

• The objective vector becomes nonpositive, and we terminate with an optimal feasible point.
• The column of the entering variable becomes nonpositive, and we terminate with an unbounded LP.

A Degenerate Pivot

Sometimes a pivot changes the basis but not the point. Here is a canonical form with respect to $\{1,2,3\}$.

The basic feasible solution is $x = (0, 2, 3, 0)$. It is degenerate: $x_1$ is basic but equals $0$.

A Degenerate Pivot

Enter $x_4$ (its objective coefficient $5 > 0$). Which variable leaves?

\[A_4 = \begin{pmatrix}1 \\ -1 \\ 2\end{pmatrix} \qquad b = \begin{pmatrix}0 \\ 2 \\ 3\end{pmatrix} \qquad b / A_4 = \begin{pmatrix}\frac{0}{1} \\ \frac{2}{-1} \\ \frac{3}{2}\end{pmatrix}.\]

A Degenerate Pivot

$x_4$ entered and $x_1$ left, so the new basis is $\{2,3,4\}$. Clearing column $4$ (pivot in row $1$) and pricing out gives:

The basic feasible solution is still $x = (0, 2, 3, 0)$ — the point did not move, but the basis and canonical form changed.

The objective value is also unchanged (still $0$). This is exactly what happens in a degenerate pivot.

One More Simplex Example

A larger LP ($4$ constraints, $6$ variables), in canonical form with respect to the basis $\{1,3,4,6\}$.

The basic feasible solution is $x = (3,0,1,2,0,2)$. Both $x_2$ and $x_5$ have positive objective coefficients.

One More Simplex Example

Let's enter $x_5$. Which variable leaves?

\[A_5 = \begin{pmatrix}1 \\ -1 \\ 0 \\ 1\end{pmatrix} \qquad b = \begin{pmatrix}3 \\ 1 \\ 2 \\ 2\end{pmatrix} \qquad b / A_5 = \begin{pmatrix}\frac{3}{1} \\ \frac{1}{-1} \\ \frac{2}{0} \\ \frac{2}{1}\end{pmatrix}.\]

One More Simplex Example

$x_5$ entered and $x_6$ left. The new basis is $\{1,3,4,6\} \setminus \{6\} \cup \{5\} = \{1,3,4,5\}$. The entering column already pivots in the last row, so no row swap is needed; we just clear column $5$.

\[\begin{pmatrix} \color{blue}1 & 1 & \color{blue}0 & \color{blue}0 & \color{red}1 & \color{orange}0 \\ \color{blue}0 & 0 & \color{blue}1 & \color{blue}0 & \color{red}{-1} & \color{orange}0 \\ \color{blue}0 & 1 & \color{blue}0 & \color{blue}1 & \color{red}0 & \color{orange}0 \\ \color{blue}0 & 0 & \color{blue}0 & \color{blue}0 & \color{red}1 & \color{orange}1\end{pmatrix} x = \begin{pmatrix} 3 \\ 1 \\ 2 \\ 2\end{pmatrix}\downarrow_{\rho_1 = \rho_1 - \rho_4}\]

\[\begin{pmatrix} \color{blue}1 & 1 & \color{blue}0 & \color{blue}0 & \color{red}0 & \color{orange}{-1} \\ \color{blue}0 & 0 & \color{blue}1 & \color{blue}0 & \color{red}{-1} & \color{orange}0 \\ \color{blue}0 & 1 & \color{blue}0 & \color{blue}1 & \color{red}0 & \color{orange}0 \\ \color{blue}0 & 0 & \color{blue}0 & \color{blue}0 & \color{red}1 & \color{orange}1\end{pmatrix} x = \begin{pmatrix} 1 \\ 1 \\ 2 \\ 2\end{pmatrix}\]

One More Simplex Example

$x_5$ entered and $x_6$ left. The new basis is $\{1,3,4,6\} \setminus \{6\} \cup \{5\} = \{1,3,4,5\}$. The entering column already pivots in the last row, so no row swap is needed; we just clear column $5$.

\[\begin{pmatrix} \color{blue}1 & 1 & \color{blue}0 & \color{blue}0 & \color{red}0 & \color{orange}{-1} \\ \color{blue}0 & 0 & \color{blue}1 & \color{blue}0 & \color{red}{-1} & \color{orange}0 \\ \color{blue}0 & 1 & \color{blue}0 & \color{blue}1 & \color{red}0 & \color{orange}0 \\ \color{blue}0 & 0 & \color{blue}0 & \color{blue}0 & \color{red}1 & \color{orange}1\end{pmatrix} x = \begin{pmatrix} 1 \\ 1 \\ 2 \\ 2\end{pmatrix}\downarrow_{\rho_2 = \rho_2 + \rho_4}\]

\[\begin{pmatrix} \color{blue}1 & 1 & \color{blue}0 & \color{blue}0 & \color{red}0 & \color{orange}{-1} \\ \color{blue}0 & 0 & \color{blue}1 & \color{blue}0 & \color{red}0 & \color{orange}1 \\ \color{blue}0 & 1 & \color{blue}0 & \color{blue}1 & \color{red}0 & \color{orange}0 \\ \color{blue}0 & 0 & \color{blue}0 & \color{blue}0 & \color{red}1 & \color{orange}1\end{pmatrix} x = \begin{pmatrix} 1 \\ 3 \\ 2 \\ 2\end{pmatrix}\]

One More Simplex Example

Price out the objective by subtracting $4$ times row $4$ (the $x_5$ row):

This is canonical form with respect to $\{1,3,4,5\}$, with BFS $x = (1,0,3,2,2,0)$ and objective value $12$.

One More Simplex Example

Only $x_2$ has a positive objective coefficient, so $x_2$ enters. Which variable leaves?

The entering column is $A_2 = \begin{pmatrix}1 \\ 0 \\ 1 \\ 0\end{pmatrix}$. The ratios $b/A_2$ are $\frac{1}{1} = 1$ (row $1$) and $\frac{2}{1} = 2$ (row $3$). The smallest is $1$, so $x_1$ leaves.

One More Simplex Example

$x_2$ entered and $x_1$ left; the new basis is $\{2,3,4,5\}$. Column $2$ already pivots in row $1$, so we just clear it from the other rows.

\[\begin{pmatrix} \color{orange}1 & \color{red}1 & \color{blue}0 & \color{blue}0 & \color{blue}0 & -1 \\ \color{orange}0 & \color{red}0 & \color{blue}1 & \color{blue}0 & \color{blue}0 & 1 \\ \color{orange}0 & \color{red}1 & \color{blue}0 & \color{blue}1 & \color{blue}0 & 0 \\ \color{orange}0 & \color{red}0 & \color{blue}0 & \color{blue}0 & \color{blue}1 & 1\end{pmatrix} x = \begin{pmatrix} 1 \\ 3 \\ 2 \\ 2\end{pmatrix}\downarrow_{\rho_3 = \rho_3 - \rho_1}\]

\[\begin{pmatrix} \color{orange}1 & \color{red}1 & \color{blue}0 & \color{blue}0 & \color{blue}0 & -1 \\ \color{orange}0 & \color{red}0 & \color{blue}1 & \color{blue}0 & \color{blue}0 & 1 \\ \color{orange}{-1} & \color{red}0 & \color{blue}0 & \color{blue}1 & \color{blue}0 & 1 \\ \color{orange}0 & \color{red}0 & \color{blue}0 & \color{blue}0 & \color{blue}1 & 1\end{pmatrix} x = \begin{pmatrix} 1 \\ 3 \\ 1 \\ 2\end{pmatrix}\]

One More Simplex Example — optimal

Price out the objective by subtracting $3$ times row $1$ (the $x_2$ row):

The objective vector has no positive entry, so this is optimal: $x = (0,1,3,1,2,0)$ with objective value $15$.

The Simplex Algorithm

The Simplex Algorithm: Formalized

Input: an LP in canonical form w.r.t. a basis $B$, and a basic feasible solution $x$.

$^*$ any choice rule works for correctness; the choice affects running time.

Correctness for Unboundedness

If the simplex method terminates with unbounded, how do we know that it is unbounded?

If the simplex method declares unboundedness, then there is a canonical form for the LP with a basic feasible solution so that $A_k$ has only nonpositive entries, and $c_k > 0$.

Claim: The vector $v$ obtained by padding $-A_k$ by 0's at all nonbasic variables except for $x_k$ and a 1 at $v_k$ is a recession direction.

Correctness for Unboundedness

For the computation, we will assume that $B = \{1,2,\dots, m\}$ and that that $k = m+1$ since we can permute the rows/columns.

Because the LP is in canonical form, we can write \[ c = \begin{pmatrix} 0 \\ c_k \\ c'\end{pmatrix}, \qquad A = \begin{pmatrix} I & A_k & A' \end{pmatrix}, \] where $A'$ is some matrix, and $c'$ is some vector.

$v = \begin{pmatrix} -A_k \\ 1 \\ 0\end{pmatrix}$, so \[ c^{\intercal}v = c_k > 0, \qquad Av = 0 \qquad v \ge 0. \]

Correctness for Optimality

A picture of optimality: the objective is maximized at a vertex, and neither edge leaving it improves the objective.

Correctness for Optimality

A picture of optimality: the objective is maximized at a vertex, and neither edge leaving it improves the objective.

Why is this picture impossible? Here, no local change can improve the objective, but nevertheless, there is an improved feasible point.

Correctness for Optimality

If simplex terminates with optimal, then we know that the LP has a canonical form with a basic feasible solution so that $c$ has only nonpositive entries.

Let $x^*$ be that basic feasible solution. We will show that $y = 0$ already certifies its optimality.

Correctness for Optimality

We work directly with the LP as given in canonical form, so the LP is

where $z$ is some constant (e.g. 5).

Correctness for Optimality

We work directly with the LP as given in canonical form, so the LP is

We can remove the constant $z$ since the optimal solutions do not change with or without it.

The termination condition is that $c \le 0$ and the fact that the LP is in canonical form imply that $c_i x_i^* = 0$ for each $i$, and in particular $c^{\intercal}x^* = 0$.

Letting $y = 0$, we get that \[Ay = 0 \ge c\qquad b^{\intercal}y = 0 = c^{\intercal}x^*. \]

Correctness for Optimality

Let's think through the logic more directly:

We have that $c_i = 0$ if $i$ is in the basis and $c_i \le 0$ if $i$ is not in the basis.

Any feasible point has all entries being nonnegative, and so $c^{\intercal}x \le 0$ for all $x \ge 0$.

On the other hand, at the basic feasible solution, $c^{\intercal}x^* = 0$, so every other feasible point has a smaller objective value than this one.

Termination

How do we know that the simplex algorithm ever terminates?

There are only finitely many vertices (because there are only finitely many bases).

Each time we change basic feasible solutions, the objective increases strictly, so we can only change basic feasible solutions finitely many times.

So we have to avoid returning to the same vertex twice; this can be accomplished with Bland's rule for selecting entering variables.

Bland's Rule

The strict-increase argument breaks under degeneracy: a pivot with step length $0$ leaves the objective unchanged, and the algorithm can cycle through a set of bases forever.

Bland's rule removes the ambiguity in which variable to pick, always breaking ties by smallest index:

• Entering: among all variables with $c_k > 0$, choose the one with the smallest index $k$.
• Leaving: among all rows attaining the minimum ratio $\frac{b_i}{A_{ik}}$, choose the one whose basic variable has the smallest index.

Theorem. With Bland's rule, the simplex method never repeats a basis, so it always terminates.

Intuitively, consistently preferring low-index variables prevents the algorithm from being "tricked" into a repeating cycle of degenerate pivots.