Theory and Practice in User-Composable Simulation Systems

Ernie Page

Jeff Opper

18 February 1999

Organization: W15DProject: 0799A56A-1A

Task 1: Application of Computability Theory to Composable Systems

Project Status

Consumed about 1.15 of 6 staff months to date Tasks undertaken during this reporting period include:

- Development of a small web repository (http://ms.ie.org/~epage/comp)

- Reviews and discussions with related efforts at SAIC, LANL

- Reviewing/recalling elements of computability theory

- Thinking about the composition problem with respect to what we understand from computability theory

Outline

Overview of Computability Theory

- Undecidability

- Computational complexity

- Class warfare in computer science: P vs NP Some computability results in simulation

- Overstreet

- Yucesan and Jacobson Application of computability theory to composability

- Categories and proofs

- What does all this mean? Discussion and direction for follow-on investigation

Sometimes “You Can’t Get There From Here”

Scenario: You represent Whizbang Computer Company. You are offered a highly lucrative contract with the following requirements:

Construct a system that, when given a Java program and the inputset for the Java program, can determine if the program will eventually stop when it is given that input. The system must work forany Java program.

What do you tell the customer?

Undecidability -- The Halting Problem

A “simple” way to show that a problem can’t be solved is to consider all possiblesolutions to the problem and show that each one contains a flaw.

Surely, such a “brute force” method is impractical to apply to the Halting Problem. Butthis is the approach used in a proof given by Allen Turing in 1936...

ProposedSolution

A

Program P

Program Input I

Halt or Loop

Suppose a solution to the Halting Problem exists. The solution, A, takes the following form:

Undecidability -- The Halting Problem

To show that A cannot exist, we first observe that both programs and input can be representedas textual data. As a special case, therefore, A must be able to determine whether anyprogram P halts when it is given a copy of P as input.

Construct another algorithm N that uses A as follows…

1. Read a program P2. Use algorithm A to determine if P halts when given P as input3. If algorithm A indicates P halts with P as input then enter an infinite loop4. If algorithm A indicates that P never halts with input P then print the

word “DONE” and stop

Algorithmfrom A

AnalyzeA’s output

(H=halt)(L=loop)

Program

P

(as program)

(as input)

Output

from A

“H” => Loop

“L” => “DONE”

Undecidability -- The Halting Problem

N is a program that halts in some circumstances and does not halt in others. Thereforethe proposed algorithm A should be able to accept N as input and determine if it halts.

But…

Suppose that A determines that N halts, then by the construction of N, N entersan infinite loop and does not halt.

Suppose that A determines that N does not halt, then by the construction of N, N prints “DONE” and halts.

Since we have arrived at these contradictions logically, it must be the case that our initialassumption was wrong and that no such algorithm, A, exists!

Sometimes “You Can Get There From Here, But it Takes a Very Long Time”

Scenario: Recently fired from Whizbang Computer Company (due to some unfortunatebusiness involving malfeasance and the Halting Problem) you now represent Harold’sHouse ‘o Multiprocessors. You are offered a highly lucrative contract by a majorPizza chain (which has invented a new high volume pizza delivery drone) with the following requirements:

Construct a system that, when given collection of addresses for pizzadelivery, will respond with the optimal (e.g. lowest fuel consumption,fastest completion time) delivery route. The system must provide theguaranteed optimal solution to any problem within 10 minutes and must accommodate problem sizes between 1 and 100.

What do you tell the customer?

Computational Complexity -- the Combinatorial Explosion Problem

A fable (from [1]):

Once upon a time, a king needed a great task performed. In response, a clever young man named Paul agreed to do the work, if the amount of payment was adequate. When the king asked Paul about his requirements for compensation, Paul responded that the king might choose between two options:

Option 1: payment could consist of one fifth of the crops produced in the kingdom for each ofthe following five years, or

Option 2: payment could be made as follows:* one kernel of corn would be given for the first square of a chess board* two kernels would be given for the second square* four kernels (twice the previous amount) for the third* eight kernels (twice the previous amount) for the fourth* and so on until payment was made for each square on the board

Unwisely, the king chose the second option. The cost of the last square alone was 263 kernels of corn (roughly 8 x 1018 kernels, or 110,000 billion bushels). The king abdicated.

Computational Complexity -- Some NumbersSource: H.M. Walker, The Limits of Computing

Value of n

Number of Operations Required

1 0.000001 0.000001 0.000001 0.000001 0.000002 0.000003 0.000001 0.000001seconds seconds seconds seconds seconds seconds seconds seconds

n n2 n3 n4 2n 3n n! nn

5 0.000005 0.000025 0.000125 0.000625 0.000032 0.000243 0.00012 0.003125seconds seconds seconds seconds seconds seconds seconds seconds

10 0.00001 0.0001 0.001 0.01 0.001024 0.059049 3.6288 2.778seconds seconds seconds seconds seconds seconds seconds hours

20 0.00002 0.0004 0.008 0.016 1.04858 58.1131 7.8218x104 3.37x1012

seconds seconds seconds seconds seconds seconds years years

50 0.00005 0.0025 0.0125 0.625 26.1979 2.3x1010 9.77x1060 2.87x1070

seconds seconds seconds seconds years years years years

200 0.0001 0.01 1.0000 1.667 4.0x1017 1.63x1034 3.0x10146 3.2x10186

seconds seconds seconds minutes years years years years

75 0.000075 0.005625 0.421875 31.6406 1.2146x109 1.95x1022 1.95x1097 1.35x10127

seconds seconds seconds seconds years years years years

* assumes that the computer is capable of performing one million steps of the algorithm per second* for perspective, the estimated age of the Universe is between 1x1010 and 2x1010 years

Computational Complexity -- More NumbersSource: Gary and Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness

Will faster computers help?

Size of Largest Problem Instance Solvable in 1 Hour

n N1 100 N1 1000 N1

n2 N2 10 N2 31.6 N2

n3 N3 4.64 N3 10 N3

n5 N4 2.5 N4 3.98 N4

2n N5 N5 + 6.64 N5 + 9.97

3n N6 N6 + 4.19 N6 + 6.29

Timecomplexity

With presentcomputer

With computer 100 x faster

With computer 1000 x faster

Generally, a problem is not considered “well-solved” until a polynomial time algorithm is known for it.

Generally, a problem is not considered “well-solved” until a polynomial time algorithm is known for it.

Computational Complexity -- The Theory of NP-Completeness

Some problems are known to be undecidable, e.g. the Halting Problem

Some problems are known to only be solvable using exponential time algorithms, e.g. Regular Expression Non-Universality Problem

Many problems are known to have polynomial time algorithms, e.g. sort n numbers, we denote these problems as belonging to Class P

From this we can construct a hierarchy of computational complexity as:

Undecidable problems

Problems requiring exponential time

Class P

Intractable problems

Computational Complexity -- The Theory of NP-Completeness

There are many problems for which only exponential (or worse) algorithms are known to exist (e.g. the pizza delivery problem noted earlier) but for which polynomial time algorithms have not been proven impossible to construct

These problems share the characteristic that they are solvable in polynomial time by a nondeterministic computer. Essentially nondeterminism provides an unlimited number of processors within which the task is partitioned. For the pizza delivery problem, if a processor exists for each possible permutation of the destinations, costs can be computed and a minimum determined in polynomial time. This class of problems is denoted NP. Note that NP contains P (if a problem can be solved in polynomial time with n processors it can be solved in polynomial time with 1 processor).

Undecidable problems

Problems requiring exponential time

?Class P

Intractable problems

Class NP

Computational Complexity -- The Theory of NP-Completeness

If all problems in NP can be translated efficiently (in polynomial time) to a problem, A, then A is said to be NP-hard. That is, A is at least as hard as any problem in NP.

If A can also be shown to be in the class NP, then A is said to be NP-complete.

Stephen Cook (1971) showed that all problems in NP were reducible in polynomial time to the SATISFIABILITY problem.

SATISFIABILITY: Given an arbitrary Boolean expression in conjunctive normalform, does the expression have a satisfying truth assignment?

Since then, many problems have been demonstrated to be NP-complete. The most comprehensive source for NP-completeness is [3].

If an efficient solution to any NP-complete problem is found then all problems in NP have an efficient solution and P=NP. If an NP-complete problem can be proven not to have an

efficient solution, then NP > P.

If an efficient solution to any NP-complete problem is found then all problems in NP have an efficient solution and P=NP. If an NP-complete problem can be proven not to have an

efficient solution, then NP > P.

Computational Complexity -- The Theory of NP-Completeness

Numerous approaches have emerged to help cope with intractability:

* Defining tractable subproblems. For example, 2-Satisfiability is in P.* Guaranteed approximate solutions. For example, a variation of the

minimum-cost spanning tree algorithm solves the travelingsalesman problem within a factor of 2 of optimal.

* Randomized algorithms.* Branch and bound techniques. For example, as employed within Dynamic

Programming* Heuristic search. For example, simulated annealing, genetic algorithms.

Some Computability Results in Simulation

Using the Condition Specification as a basis, Overstreet demonstrates that the following problems are undecidable:

* whether any model specification is finite (i.e. an implementation will run to termination)* whether any two model actions are order independent* whether any model specification is complete* whether any model specification is minimal* whether any two model specifications are functionally equivalent

C. Michael Overstreet. “Model Specification and Analysis for Discrete Event Simulation,” Ph.D. Dissertation, Dept. of Computer Science, Virginia Tech, Blacksburg, VA, December 1982.

Some Computability Results in Simulation

Using the Event Graphs as a basis, Yücesan and Jacobson demonstrate that the following problems are NP-hard:

* whether or not any given state will occur during an execution of the model* determining the existence of, or ruling out the possibility for, simultaneous events* determining whether the model implementation satisfies the model specification* determining whether an execution can reach a state in which the termination condition

has not been met and no events are on the event list

Enver Yücesan and Sheldon H. Jacobson.“Building Correct Simulation Models is Difficult,” In: Proceedings of the 1992 Winter Simulation Conference, pp. 783-790, Arlington, VA, December 13-16 1992.

Enver Yücesan. “Simulation Graphs for the Design and Analysis of Discrete Event Simulation Models,” Ph.D. Dissertation, Cornell University, Ithaca, NY. 1989.

Enver Yücesan and Sheldon H. Jacobson. “Computational Issues for Accessibility in Discrete Event Simulation,” ACM Transactions on Modeling and Computer Simulation, 6(1):53-75, January 1996.

Sheldon H. Jacobson and Enver Yücesan. “Intractability Results in Discrete-Event Simulation,” Operations Research, 29(3):353-369, 1995.

Composability and Computability

We are interested in user-directed composition, e.g.

* a user provides a question (in the form of a set of modeling objectives) toa web browser, the web is searched for suitable candidate models, and thebest (or a “good enough”) solution is automatically constructed and made available on the desktop. (e.g. the web-based simulation vision)

* a man walks into a room and says, “computer, I want to see a tank with wings andthe capability for water-based take-off and landing.” And it happens. (e.g. theholodeck vision)

We suggest that such systems represent an “ease of use” for simulation that is a worthy goal.

We are interested in the question of how composition can be employed to realize these systems. Particularly, we are interested in determining “what price, ease of use?”

Composability and Computability

Are there risks associated with building simulation models through composition?

We suggest that both “horizontal” and “vertical” dimensions to the composition problem exist.

Horizontal composability involves coupling of peer components, e.g. a ground combat model and an air combat model. Composing in this dimension introduces (encourages?) multi-resolution modeling and the difficulties attendant with it.

Veritical composability involves abstraction through aggregation, e.g. creating a tank battalion from a collection of tank platoons. Such abstraction places at risk the identification of the most efficient (or even a correct) solution.

Abstraction through Aggregation: An Example

Objective: Develop a statistically valid model of traffic flow in the city of Albuquerque

Assumption: High-fidelity model initially assumed to be superior to aggregate models (flow equations, etc.)

First Result: LANL developed detailed models of individual automobiles and driver behaviors that were used to evaluate traffic patterns within the city.

Model results showed no correlation with actual traffic patterns.

Second Attempt: LANL simplified the model and used cellular automata to describe vehicle movements based upon a simple rule set.

New model was easily calibrated to established traffic patterns.

Abstraction through Aggregation: Another Example

Objective: Determine the average utilization of a bank teller during a peak situation, e.g. lunchtime

Observation: It is generally well known that arrivals in systems such as these follow a Poisson process (modeler need only identify the mean inter-arrival time)

Two models:

Question: Which model is more likely to be valid? The “low fidelity” model, A, or the “high fidelity”(entity-based) model, B?

Answer: If individual customer decisions do not produce Poisson arrival rates (with appropriatemean) then model A is superior!

A: arrivals modeled through sampling from a statistical distribution;model contains only a “high fidelity” representation of the teller

B: model contains “high fidelity” representations of teller and customers; customer arrivals (or decisions to balk) are modeled via autonomous decision rules embedded in customer objects

Composability and Computability

In addition to the risks attendant with the horizontal and vertical dimensions of composability, there seems to be an obvious scalability limit to the general reusability problem (we view composability as a special case of the general reusability problem):

* As the number of candidates for reuse (composition) becomes large, the benefits of reuse(composition) become negated by the costs ($ and time) of storage, organization and retrieval.

* Consider the web, for example: the more information it contains, the harder it becomes to find what you need and the less useful the web becomes.

* If the web-based simulation vision is realized--and if publishing models becomes profitable, it will be realized--the proliferation of models will mirror the proliferation ofinformation on the web today.

But composition dictates not only identifying relevant candidates from (possibly massive) component repositories but also answering the following:

* Does a combination of components exist that satisfies the modeling objectives?* If so, identify the “best” solution in a reasonable time. If not, how closely can the

objectives be met?

Composability and Computability

Determining that a collection of components satisfies a requirement (objective) might be accomplished in any number of ways, including:

* determination made strictly on the basis of component “metadata”* determination made by modeling/approximating component interactions* determination made by constructing the composition and observing the result

We observe that “ideally”, we would like to make the determination using the first approach (since it is computationally the simplest).

Intuitively, we expect that even in the determination-by-metadata case determination is intractable due to combinatorial explosion. For any feature for which a variety of components are candidates, the complexity of determining the best combination could be O(2n) (if we have to examine all possible combinations).

Composability Formalized

Let O = {o1, o2, … ,on} be a set of system objectives. Informally, we say that a system, S, satisfies or meets O if and only if each of the objectives in O are met. (A variety of fomalizations for satisfaction are possible here.)

Let C = {c1, c2, … ,cm} be a set of components located in a repository R. A system, S, consists of one or more of these components. If |S| > 1 then we say that S is a composed system. We say that the composition of two components, ci and cj is valid if the mechanisms necessary to make make these two components interact are available. This may mean that c i and cj are present in binary format, or that rules for their joint compilation are specified, or that the components can communicate via a common interface. We will allow the notion of “valid composition” to be context dependent.

COMPOSABILITYINSTANCE: A set O of objectives and a collection C of components.QUESTION: Is there a valid composition that meets the objectives stated in O?

Conjecture: COMPOSABILITY is NP-complete

Composability Formalized

We have difficulty establishing that COMPOSABILITY is NP-complete because it is not clear that COMPOSABILITY is in NP (we may be able to guess a solution, but some objectives may not be verifiable in polynomial time, e.g. those results established by Overstreet, Jacobson and Yücesan).

For a set of objectives, O, if it is possible to verify in polynomial time that each objective in O is satisfied, then we say that the system objectives are bounded. If one or more objectives cannot be verified in polynomial time, we say the objectives are unbounded.

Let ¤ denote the composition relation. We denote the composition of component a and component b as: a ¤ b. Generally, compositions may display a type of emergent behavior, that is, the capability of a composition may not be the simple sum, product, or union of the capabilities of the individual components within the composition. We state this formally in terms of system objectives. Let denote the satisfies operator. If a o we say that “a satisfies o”. If a o and b o but (a ¤ b) o then we say the composition is nonlinear. If a o and b o and (a ¤ b) o then we say the composition is linear.

On this basis we define four variants of the COMPOSABILITY problem.

Composability Formalized

UNBOUNDED NONLINEAR COMPOSABILITYINSTANCE: A set O of unbounded objectives and a collection C of components demonstrating nonlinear composition.QUESTION: Is there a valid composition that meets the objectives stated in O?

BOUNDED NONLINEAR COMPOSABILITYINSTANCE: A set O of bounded objectives and a collection C of components demonstrating nonlinear composition.QUESTION: Is there a valid composition that meets the objectives stated in O?

UNBOUNDED LINEAR COMPOSABILITYINSTANCE: A set O of unbounded objectives and a collection C of components demonstrating linear composition.QUESTION: Is there a valid composition that meets the objectives stated in O?

BOUNDED LINEAR COMPOSABILITYINSTANCE: A set O of bounded objectives and a collection C of components demonstrating linear composition.QUESTION: Is there a valid composition that meets the objectives stated in O?

Composability Formalized

Theorem: BOUNDED LINEAR COMPOSABILITY is NP-complete.

Sketch of the Proof: BLC is in NP since a nondeterministic algorithm may guess a composition and, since the objectives are bounded, may verify in polynomial time that the composition meets the objectives.

We transform SAT to BLC. First we observe that attainment of objectives is a conjunctive property. Further we observe that there may exist numerous ways to meet an objective. We define a set, P, of such satisfying “properties” for each objective. Selection of a component introduces a set of properties. A composition is well-formed if it contains no conflicting properties.

Let U and C represent an arbitrary instance of SAT. We construct an instance of BLC by mapping elements of U to elements of P, and for each C we form a set P’ that maps to an objective in O. We assume without loss of generality that each property is related to one and only one component. We construct a set of candidate components such that one introduces property p and one introduces property “not p” for each p in P. From this construction it is straightforward to show that O is satisfied if and only if C is satisfied.

What Does All This Mean? (Some Preliminary Conclusions)

The fact that BLC is NP-complete means that even if we have perfectly captured requirements and perfectly rich component metadata and we are able to search and identify candidate components quickly, the act of finding the best composition for the problem at hand is intractable (even for as few as 50 candidates). Therefore it seems we should focus on the application of approximate techniques that help quickly identify solutions that are “good enough”. We are currently developing a research proposal for internal MITRE funding to address these issues.

The results from Overtsteet, Jacobson and Yücesan apply to composable systems as well. Algorithms to answer the following questions (in the general case) cannot exist or cannot be efficient:

* Do I have the minimum set of components necessary to solve my problem?* Will my composed simulation run to completion?* Is my composed system complete?* Will my composed system meet a specified set of performance criteria?

It is arguable that none of the fundamentally hard problems associated with simulation (or computer-based systems generally) are solved by composability.

Direction for Follow-On Work

At this point we have introduced more questions than we have answered. What can we say about the complexities of the classes of composability outside of BLC? Are other classes of the problem worth defining? Our formulation was based on the situation where all possible combinations of a group of components must be considered. Is this reasonable? Are other reasonable formulations possible, e.g. those built on the product rule or on combination/permutation rules? What are the complexities of these formulations?

In the initial plan, Task 2 (the second staff month) was defined to investigate notations for specifying composable systems. We think that the current line of investigation merits another staff month of effort as Task 2.

Discussion?

Sources

[1] Henry M. Walker. The Limits of Computing. Jones and Bartlett Publishers. Boston, MA.1994.

[2] A. K. Dewdney. The Turing Omnibus: 61 Excursions in Computer Science. Computer Science Press, Rockville, MD. 1989.

[3] Michael R. Garey and David S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Company, New York, NY. 1979.

[4] Harry R. Lewis and Christos H. Papadimitriou. Elements of the Theory of Computation. Prentice-Hall, Englewood Cliffs, NJ. 1981.