Hoffman's Circle Untangled

Hoffman's Circle UntangledAuthor(s): Jon LeeSource: SIAM Review, Vol. 39, No. 1 (Mar., 1997), pp. 98-105Published by: Society for Industrial and Applied MathematicsStable URL: http://www.jstor.org/stable/2133007 .

Accessed: 14/06/2014 17:48

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Society for Industrial and Applied Mathematics is collaborating with JSTOR to digitize, preserve and extendaccess to SIAM Review.

http://www.jstor.org

This content downloaded from 185.2.32.58 on Sat, 14 Jun 2014 17:48:30 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=siam

http://www.jstor.org/stable/2133007?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


SIAM REV. ? 1997 Society for Industrial and Applied Mathematics Vol. 39, No. 1, pp. 98-105, March 1997 006

HOFFMAN'S CIRCLE UNTANGLED*

JON LEEt

For Alan Abstract. In 1951, Alan J. Hoffman provided the first example that demonstrated that Dantzig's

simplex method for linear programming can cycle (unless special precautions are taken) This paper describes a geometry for understanding Hoffman's example-something that has been lacking for over two-fifths of a century.

Key words. linear programming, simplex method, cycling

AMS subject classification. 90C05

PII. S0036144595292991

1. Introduction. In 1947, George B. Dantzig designed the simplex method for solving linear programs (see Dantzig [5]). We consider linear programs in the form

min z p: s.t. Ax =b

P: ~~~~~~T C X= Z,

x > 0,

where A is an m x n matrix of rank m, b is an m-vector, and c and x are n-vectors. The simplex method relies on the idea of a "basic feasible solution." At an iteration, the indices {1, 2,.. . ., n} of x are partitioned into an ordered basis /3 = (31, p2 ...... *3m)

and nonbasis i1 = (?1j, q2.... 71n-m) in such a way that A,3 is nonsingular and x,3 = A-lb > 0. Then, letting x,: 0, x is a basic feasible solution to P. We will assume that P has a feasible solution; it is an exercise to show, then, that P has a basic feasible solution. Obviously, there are only a finite number of basic feasible solutions.

The simplex method proceeds to the next basic feasible solution by pivoting-that is, switching one index from /3 with one index from r1. Index j E i is eligible to enter the basis if cj := cTdi < 0, where di E JRn is defined by d& := 1, d3 := -A- A and d&\j 0. It is common to refer to c; as the reduced cost of xj. It is an exercise to show that if no index is eligible to enter the basis, then the current basic feasible solution is optimal.

After choosing the entering index j, index i is eligible to leave the basis as long as the ratio -xl/di = A := min{-xk/dk : dk < 0}. If the problem has a finite optimum, then there is always an index that is eligible to leave the basis. The new basic feasible solution will have lower objective value than the previous solution as long as the variable with the entering index attains a positive value in the new solution. This happens precisely when A > 0. (The value of xj after the pivot is A and the change in the objective value is A * c;.) If we are fortunate enough for this to happen at every iteration, then we will produce a sequence of basic feasible solutions with ever decreasing objective value. Eventually, the method would have to terminate, either finding an optimal solution or demonstrating that there is no finite optimum. If, at

*n

*Received by the editors August 18, 1995; accepted for publication September 12, 1995. This research was supported in part by NSF grant DMI-9401424.

http://www.siam.org/journals/sirev/39-1/29299.html tDepartment of Mathematics, University of Kentucky, Lexington, KY 40506-0027 (jlee@

ms.uky.edu, http://www.ms.uky.edu/-jlee).

98



CLASSROOM NOTES 99

some iteration, the variable with the entering index has value 0 in the new solution, then we have encountered degeneracy.

Method was an apt choice of word, at the time, since the simplex method is not guaranteed to terminate if degeneracy is encountered. What makes the simplex method an algorithm is a refinement of the pivot choice rules that guarantees that a basis is never repeated, even under degeneracy. Dantzig referred to a (then hypothet- ical) sequence of iterations repeating a basis as a circle. Nowadays, cycle is a more common term.

Refinements of the simplex method that avoid such circles have been proposed, thus turning the simplex method into a bona fide algorithm. Dantzig described the first such refinement, based on an algebraic perturbation of the vector b, in his linear programming course at the Graduate School of the U.S. Department of Agriculture in the fall of 1950. (Charnes [3] and Dantzig, Orden, and Wolfe [7] published proofs that such refinements avoid circles.) Many years later, Bland [2] introduced a combinatorial refinement that avoids circles.

In 1951, Alan J. Hoffman constructed the first example of a circle, demonstrating the theoretical need for a refinement of the pivot choice rules of the simplex method-fortunately, Dantzig had already described his refinement that avoids circles. Hoffman's example has 3 equations and 11 variables (though the essential aspects of Hoffman's example require only 2 equations and 10 variables). Later, E.M.L. Beale [1] gave another example of a circle with 3 equations and only 7 variables. Marshall and Suurballe [10] eventually showed that the smallest example of cycling has 2 equations and 6 variables. (They also provide an example with 3 equations and 6 variables and one with 4 equations and 7 variables.) None of these examples is as mysterious as Hoffman's.

Hoffman's example originally appeared in a National Bureau of Standards Report (see Hoffman [9]). It is most readily available in Dantzig's book [6]. Let

XO 1 0 0 0 X1 0 1 0 0 X2 0 0 1 0 x3 0 cos 0 tan G sin 8 cos 8-1

x4 0 -w cos 6 cos 0 w (AT c) x5 0 cos 20 tan 0 sin29 0

x6 0 - 2w cos Cos 20 2w X7 0 cos 20 -2sin2O 4 sin2 o xg 0 2w cos2 0 cos 20 -2w cos 20 x9 0 cos 0 -tan 0 sin 0 4 sin2 8 x1o 0 w cos 0 cos 0 w(1-2 cos 0)

and

bT:=(1 0 0),

where 0 2ir/5 and w > (1 - cos 0)/(1 - 2 cos 0). The circle consists of the repeating sequence of bases

/ 3-(0, j-1 (mod 10) + 1, j (mod 10) + 1) for j = 1, 21 ....

We note that this linear program has x = (1) as both its unique optimum and its only basic feasible solution. The simplex method is pivoting to verify the optimality of this solution.



100 CLASSROOM NOTES

How did Hoffman construct this example? In his NBS report, Hoffman states the following:

"Finally, we regret that we are unable at this date to recall any details of the considerations that led to the construction of the example, beyond the fact that the geometric meaning of [the pivot rules of the simplex method] was very much in the foreground."

In Beale's paper we find the following:

"Unfortunately, the geometric motivation behind Hoffman's example is not easy to grasp."

More recently, I had the following e-mail exchange with Hoffman:

Date: Wed, 3 Aug 1994 13:48:46 -0500 (EDT) To: alan hoffman <hoffa?watson.ibm.com> From: jon lee <[email protected]> Subject: circling

hi alan. something has been bothering me for a while, so i thought that i would resolve it by going to the source. in dantzig's linear programming book, he refers to the repetition of a basis by the simplex method as a 'circle' rather than the more common 'cycle' (he prefers to refer to an iteration as a cycle); anyway, the term 'circle' seems rather appropriate when he gives your example where the entries in the simplex table (i prefer table to tableau, but that is another mat- ter) are ordinary trigonometric functions! and now my question: how did you concoct this example? it is not transparent to me. surely you remember; it was only 43 years ago! - jon

Date: Mon, 8 Aug 94 12:24:35 EDT To: [email protected] From: [email protected] Subject: circling

Two months after I made up the example, I lost the mental picture which produced it. I really regret this, because a lot of people have asked me your question, and I can't answer. best regards to carl. alan

This paper describes a geometry for understanding Hoffman's example. It turns out that Hoffman's circle has a description which is appealing from an algebraic as well as a geometric viewpoint. In section 2, we motivate Hoffman's circle purely algebraically, and in section 3, we describe a geometry for Hoffman's circle.

2. An algebraic derivation of a family of circles. Let 0 = 2ir/k, where k > 2 is an integer to be determined. The idea is to use the symmetry of the geometric circle and complete a simplex circle in 2k pivots. Let y > 0 be a constant to be determined. Let



CLASSROOM NOTES 101

Let

(cosO -sinO0

R=sinO 0CosO J Then, for j = 1, 2, ... , 2k, let

Aj f R(j-1)/2Al for odd j, RU-2)/2A2 for even j,

and let c 1 := 1-aj - ()a2j. It is apparent that for odd j, Aj is a rotation of A1 by (j - 1)7r/k radians, and

for even j, Aj is a rotation of A2 by (j - 2)ir/k radians. We can view the k points (Aj) with j odd as being on the intersection of the (right) cylinder

(a2) E 3: a 2 +a 21}

with the plane

ll:= t a2 ER3 : ai+( )a2+z=l}.

The k points (Aj) with j even are on the intersection of the (right) cylinder

(a2) E R3: a2 +a2=2}

with the plane II. Moreover, for each even j, Aj can be thought of as a rotation of Ai-, by ir/2

radians followed by a scaling by -y. This means that the angle between the basic columns alternates between ir/2 and ir(k - 4)/2k as we pivot around the circle.

Now, we want to arrange things so that with respect to the basis

(j - 1 (mod 2k) + 1,j (mod 2k) + 1)

the index j + 1 (mod 2k) + 1 is a candidate to enter the basis and j - 1 (mod 2k) + 1 is a candidate to leave that basis. It is easy to check that { sec OAj-1 (mod 2k)+1 - 7 tan OAj (mod 2k)+1 for even j,

A3+i (mod 2k)+1 = cosOAj-1 (mod 2k)+1 + 1 sin OA (mod 2k)+1 for odd j.

The leaving condition requires that cos 0 > 0 (and, redundantly, that sec 0 > 0); hence we must choose k so that 0 < 0 < 2 or k > 5. 2

Furthermore, we can make the reduced cost calculation

(1-sec0+-ytan0 forevenj, Ej+1 (mod 2k)+i 1 - cos0- 1 sin0 for odd j.



102 CLASSROOM NOTES

The entering condition requires that the reduced cost be negative; hence we require that 1 - sec 0 + y tan 0 < 0 and 1 - cos 0- 1 sin 0 < 0. The dominating restriction on

-y y is

sin 0 1-cosO'

which, for the choice of ay = cot0, yields the redundant inequality cos0 < 1. The result is that we have constructed a circle for every integer k > 5.

The choice of k = 5 and -y = cot 0 yields, essentially, the circle that Hoffman presented. There are two minor differences. The first difference is Hoffman's use of the parameter w. In the early days of linear programming, it seemed reasonable to follow Dantzig's rule: Choose j E r to enter the basis if -j is minimum among all nonbasic indices that are eligible to enter the basis. If there are ties, it was suggested that we choose the minimum j for which -c is minimized. If there are several indices eligible to leave the basis, it was suggested that we choose the eligible index i that appears leftmost in f. The constant w in Hoffman's circle is really just a scaling parameter that is used to make the circle obey these rather arbitrary refinements of the pivot rules of the simplex method. Nowadays, we are not so set on any particular refinement, so we do not mind rescaling the problem. In order to make our geometric picture more appealing, we apply the following scaling to Hoffman's example:

A b dia (1l 17w cot 0, 1) (A b diag(i, 1 1 1 w-l,1 w-l,l,w-l,l,w-l,l).

This scaling eliminates the parameter w. The other difference is even more minor. We have suppressed the first variable and first constraint of Hoffman's data since they are not involved in the circle their apparent purpose being to avoid a completely degenerate example.

3. The column geometry of Hoffman's circle. Most textbooks on linear programming describe degeneracy and its resolution. Although most textbooks give geometric pictures of degeneracy, no geometric pictures of circles are provided. The main reason for this must be that most texts dwell on geometric pictures of the polyhedron of feasible solutions of P, projected onto the space (Rn-m) of the initial set of nonbasic variables. In this picture, the simplex method moves from extreme point to extreme point along edges of the feasible region. This is a fine picture when n - m = 2, but all examples of circles satisfy n - m > 3 (see [10]). Marshall and Suurballe give two examples with n - m = 3, so there is potential for a revealing three-dimensional picture in this geometry. Hoffman's example with n - m = 8 was certainly not motivated by this geometry.

Another possible geometry is the space of variables y E 'm of the dual linear program

max bT y s.t. AT y < cT

In fact, it was this geometry of the dual, together with Hoffman's example, that motivated Beale's investigations. Based on the remarks of Beale and Hoffman, it is clear that this geometry of the dual was not the geometry that motivated Hoffman. Marshall and Suurballe also worked with this dual geometry.

In addition, Marshall and Suurballe presented an interesting geometric picture of a circle of the "dual simplex method" for an example with 3 equations and 6



CLASSROOM NOTES 103

variables. Their picture involves a two-dimensional depiction of the columns of [Alb] using barycentric coordinates (in their example, [Alb] has constant column sums).

There is another geometry for the simplex method. In fact, it is closely related to the geometry that gave rise to the name simplex method. The so-called column geometry of the simplex method lives in Rm+1. We consider the n points ( cA) C RM+l

and the requirement line {(b) E Rm+1 : Z E}. A basis 3 determines an m- dimensional simplicial cone

C3 {(b) E Rm+l b) = (A')x/3 for some x13 E }.

If 1 determines a basic feasible solution with objective value z, then the requirement line pierces C01 at the single point (b). Now, the linear span of C01 is a hyperplane in Rm+1. We let H,B denote the open half-space, bounded by the linear span of C01, that contains the half of the requirement line having z < z.

Recall that the condition for j E r to be eligible to enter the basis is that Cj :=cTdj = -cpA-1Ai < 0. This has a geometric interpretation in the column geometry. The expression Aj = A13(A 'Aj) describes how Aj is a linear combination of the columns of A4. Using the same linear combination, we can interpolate an objective coefficient for xj, namely, Cj :=cT(A-1Aj). Therefore, Cj =Cj - j, and the condition for j to be eligible to enter the basis amounts to the objective coefficient being less than the interpolated value. Or, geometrically, the point (,Aj) falls directly below the point () . Said a slightly different way, the nonbasic index j is eligible to enter the basis if (iAj) lies in Ho.

Now, C0uj is an (m + 1)-dimensional simplicial cone which has C01 as a facet. If the half of the requirement line with z < z is wholly contained in C13uj, then P has no finite optimum. Otherwise, the lowest point on the requirement line that is in C)3uj is contained in some facet C03\iuj of C13uj for some i E 1. Any such i is a candidate to leave the basis.

In fact, it was in the setting of the column geometry that Dantzig proposed the simplex method. Current popular linear programming books such as Chvatal [4] and Schrijver [12] do not even mention the column geometry of the simplex method, while some older books such as Dantzig [6, pp. 160-166], Gass [8, pp. 88 89], and Murty [11, pp. 132-135] do include the topic.

In this column geometry, it is instructive to see a picture of the convex hulls of the three points:

( O A) ( C1 A ( 82)

for each basis (k1, 02) of our circle with k = 5 and y = cot 0 (see Figures 1 and 2). Each such triangle suggests its linear and conical span. In the appendix, I have provided Mathematica (see Wolfram [13]) commands that provide an animated version of Figure 1. This is very useful for visually checking that the pivot rules of the simplex method are respected at each pivot.

A physical model of a "Hoffcicle" is available from the author.

Appendix. Below appear Mathematica commands to animate Figure 1.

<<Graphics 'Animation'; t=2Pi/5; c=Cos[t]; s=Sin[t]; n=69; e=.75;



104 CLASSROOM NOTES

FIG. 1. k = 5, c = otO.

3~~~ 5

7a

9

FIG. 2. A view from above.

M={{fc,-s,o},{8,C,o}, {(C-1)/C,8/C, i}};

y={{O},{C0t[t]}3,{O}};

T=Partition[Flatten[{x,y,m.x,m.y,m.m.x,m.m.y, m.m.m.x,m.m.m.y,m.m.m.m.x,m.m.m.m.y,x}) ,{3}, {3}J;

ShowAnimation[Table[Graphics3D[Table--[({GrayLevel[e],Polygon[{{O,0,0}, T[(iJ],T[[i+lJJ}J},{i, 10}],BoxRatios->{2,2,2}, SphericalRegion->True, ViewPoint->{Sin[2Pi*t/n],Cos[2Pi*t/nJ,0}, Lighting->False]J,{t,0,n}JJ



CLASSROOM NOTES 105

REFERENCES

[1] E. M. L. BEALE, Cycling in the dual simplex algorithm, Naval Res. Logist. Quart., 2 (1955), pp. 269-275.

[2] R. G. BLAND, New finite pivoting rules for the simplex method, Math. Oper. Res., 2 (1977), pp. 103-107.

[3] A. CHARNES, Optimality and degeneracy in linear programming, Econometrica, 20 (1952), pp. 160-170.

[4] V. CHVATAL, Linear Programming, W. H. Freeman, San Francisco, CA, 1983. [5] G. B. DANTZIG, Maximization of a linear function of variables subject to linear inequalities,

in Activity Analysis of Production and Allocation, T. C. Koopmans, ed., John Wiley, New York, 1951, pp. 339-347.

[6] G. B. DANTZIG, Linear Programming and Extensions, Princeton University Press, Princeton, NJ, 1963.

[7] G. B. DANTZIG, A. ORDEN, AND P. WOLFE, The generalized simplex method for minimizing a linear form under linear inequality restraints, Pacific J. Math., 5 (1955), pp. 183-195.

[8] S. I. GASS, Linear Programming, Volume 4, McGraw-Hill, New York, 1975. [9] A. J. HOFFMAN, Cycling in the simplex algorithm, National Bureau of Standards Report 2974,

1953. [10] K. T. MARSHALL AND J. W. SUURBALLE, A note on cycling in the simplex method, Naval

Res. Logist. Quart., 16 (1969), pp. 121-137. [11] K. G. MURTY, Linear and Combinatorial Programming, John Wiley, New York, 1976. [12] A. SCHRIJVER, Theory of Linear and Integer Programming, John Wiley, New York, 1986. [13] S. WOLFRAM, Mathematica: A System for Doing Mathematics by Computer, Second Edition,

Addison-Wesley, Reading, MA, 1991.



Date post:	12-Jan-2017
Category:	Documents
Upload:	jon-lee
View:	215 times
Download:	2 times

Hoffman's Circle Untangled

Documents