Preliminary Thoughts Towards a Practical Theory of Reformulationfor Reasoning about Physical Systems
Berthe Y. Choueiry, Sheila McIlraith~ Yumi IwasakiTony Loeser, Todd Neller, Robert S. Engelmore, and Richard Fikes
Knowledge Systems LaboratoryStanford University
Stanford, CA 94305-9020
Abstract
In this paper, we provide a practical framework forcharacterizing, evaluating and selecting reformulationtechniques for reasoning about physical systems, withthe long-term goal of automating the selection andapplication of these techniques. We view reformula-tion as a mapping from one encoding of a problemto another. A problem solving task is in turn accom-plished by the application of a sequence of reformula-tions to an initial problem encoding to produce a finalencoding that addresses the task. Our framework pro-vides the terminology to specify the conditions underwhich a particular reformulation technique is applica-ble, the cost associated with performing the reformula-tion, and the effects of the reformulation with respectto the problem encoding. As such it provides the vo-cabulary to characterize the selection of a sequence ofreformulation techniques as a planning problem. Ourframework is sufficiently flexible to accommodate pre-viously proposed properties and metrics for reformu-lation. We have used the framework to characterize avariety of reformulation techniques, three of which arepresented in this paper.
1 IntroductionReformulation plays an important role in various intel-lectual activities and is ubiquitous in reasoning aboutphysical systems. Reformulation improves the effec-tiveness of a mental or computational problem-solvingprocess by recasting a problem into a new one that istailored to a given task. The selection of reformulationtechniques must be carried out relative to a problem-solving task. In this paper we examine the role of re-formulation in reasoning about physical systems, andprovide a practical framework for evaluating various re-formulation techniques applicable to this class of prob-lems.
Informally, we define reformulation to be a mappingfrom one encoding of a problem to another. A problem-solving task is accomplished by the application of aselect sequence of reformulations to an initial problemencoding to produce a final encoding that addresses
* and Xerox Palo Alto Research Center, 3333 CoyoteHill Road, Palo Alto, CA 94304
the task. We use the term reformulation to subsumethe notions of abstraction and approximation, therebyavoiding any lexical implication that the mapping gen-eralizes or simplifies the domain theory. Given an en-coding of a problem and a reasoning task, one maychoose to reformulate for any of the following reasons:1. Engine-driven problem re-encoding: enabling the use
of a particular reasoning engine by satisfying its in-put requirements, either because no engine existsto address the initial problem, in order to improvethe performance of problem-solving, or to reduce thecost of the reasoning.
2. Cognitive insight: improving the user’s understand-ing of the problem or solution space.In order to develop a framework with sufficient de-
tail to compare reformulation techniques, we focus ona specific class of problems, namely reasoning aboutphysical systems. We require that the behavior ofthe physical system be expressible as a set of lumped-parameter hybrid (continuous and discrete) models,containing algebraic or differential equations, but prob-lem solving need not be restricted to a direct manip-ulation of equations. Finally, we require that the taskbe motivated by a specific query, thus constraining thecomputational machinery necessary to carry it out.
The long-term goal of our research is to developan automatic task-driven capability that selects andapplies appropriate reformulation techniques in thecourse of modeling and analyzing the behavior of phys-ical systems. The contribution of this paper is a prac-tical framework for evaluating specific reformulationtechniques with respect to the restricted class of prob-lems we described above. The motivation for develop-ing such a framework came from the observation thatmuch of the previous work on reformulation (includingabstraction and approximation) was either too specificor too general to be of practical use in developing au-tomated reformulation mechanisms. Our frameworkprovides a significant step towards this long-term goalby defining general criteria for understanding the prop-erties of various reformulation techniques.
The paper is organized as follows. Section 2 intro-duces our conception of the processing stages involved
in reasoning about physical systems, setting the con-text for our reformulation framework. Section 3 intro-duces the framework itself and presents a set of evalua-tots for assessing the effects of reformulation. Section 4applies the framework to three examples of reformula-tion techniques previously described in the literature.Section 5 discusses related work, and Section 6 con-cludes with a brief summary and outlines directionsfor future research.
2 Reasoning about Physical SystemsIn this section we describe the various processes thatmay be executed in reasoning about physical systems.We illustrate these processes in terms of several ex-amples drawn from the literature. Starting with a de-scription of the task of interest, we perceive the entireendeavor as a progression through the three process-ing stages illustrated in Fig. 1, namely: the model, theequation and the solution processing stages. Identify-ing these stages has proved instructive in distinguish-ing and situating the various reformulation techniquesuseful for reasoning about physical systems.
Figure 1: Reasoning about physical systems. Stages andtheir corresponding processes.
Z2 QR-98
Task description. Reasoning about physical systemsbegins with a task description specified by the fol-lowing four elements: the domain theory, the sce-nario, the query and the modeling assumptions. Thedomain theory is a corpus of knowledge correspond-ing to heterogeneous, possibly redundant or contra-dictory descriptions of a physical system, includinglogical statements, symbolic equations, numeric pa-rameters. The scenario is a description of a partic-ular problem instance (e.g., a set of system compo-nents and their physical structure, and the initialconditions of the system). The query is an explicitspecification of the user’s question (e.g., variables,their quantitative or qualitative values, direction ofchange at specific time points). Finally, the model-ing assumptions include assumptions that the prob-lem solver may make in order to broadly delimit thescope of the answer to the query (e.g., the temporaland physical extent of its coverage, granularity).
Model processing. Given a task description, themodel building process assembles the relevant as-pects of the domain theory to produce a model,which is an instantiation of a subset of the domaintheory that is both consistent and sufficient to ad-dress the query. A model at this point often consistsof knowledge of the physical structure (componentsand their topology, for example) as well as knowl-edge of the relevant physical phenomena (includingthe conditions under which they are active), in con-trast to the purely mathematical model of the fol-lowing stage. A typical example of a model build-ing process is compositional modeling as in [4; 15;12] and in the modeling algorithm of TRIPEL [19].This process can be followed by a model reformu-lation process. Model reformulation may involvestructural consolidation [23], time scale selection[19], or simplification [16] or expansion of a modelthrough addition, deletion, or replacement of a de-scription of components or phenomena.
Equation processing. Equation building producesan equation model1 either directly from a task de-scription or through reformulation of a model intoa set of mathematical equations describing the be-havior of the system. For example, the QualitativePhysics Compiler [5] converts a model expressed inQP Theory [6] into a set of qualitative differentialequations.An equation reformulation process may then be car-ried out. An equation reformulation is primarily mo-tivated by a desire to transform the present equationmodel into a form that is amenable to a particularproblem-solving engine. Many reformulations oper-ate at this stage, such as mapping polar to Carte-
tWe distinguish between non-equational models andequational ones in order to be able to represent the var-ious possibilities for manipulating models reported to datein the literature for reasoning about physical systems.
sian coordinates, mapping a time domain to a fre-quency domain, reformulating ordinary differentialequations as qualitative differential equations, drop-ping insignificant terms, linearizing, and aggregatingnearly decomposable systems.
Solution processing. The solution building processis often a problem-solving engine acting on eitherthe model (e.g., QPE [7]) or the equations (e.g.,QSIM [11] and Mathlab@) to produce one or moresolutions to the query.
A solution reformulation process may subsequentlybe performed to enhance cognitive insight. Exam-ples of such reformulations are summarization [14]and explanation by generation of active documen-tation [9]. Solution reformulation may also be ap-plied for engine-driven problem re-encoding. For ex-ample, Clancy and Kuipers [1] interleave a QSIMsimulation with the aggregation of partial solutionscorresponding to chatter in a qualitative simulation.In so doing, they significantly improve the overallperformance of QSIM on their problem.
Note that reasoning need not necessarily transitionthrough every processing stage, nor through every pro-cess within a stage. For example, a particular problem-solving task may go from task description directly tothe equation processing stage, or from the model pro-cessing stage directly to the solution processing stage.Moreover, a problem-solving task may loop any num-ber of times through one or more of the individualprocesses. In the example of solution reformulationreported above, Clancy and Kuipers [1] loop over thesolution building and solution reformulation processes.During the simulation of the behavior of a dynamicalsystem, the operating conditions may change as a re-sult of system dynamics. The model used for simu-lation must then be updated to comply with the newassumptions. In such situations, the reasoning processmay involve a cycle of model processing, equation pro-cessing and solution processing as the system sequen-tially transitions from one discrete state to another.
In Section 4, we provide one example of reformula-tion at each of the model, equation, and solution pro-cessing stages.
3 Proposed framework
In this section we introduce a framework and a termi-nology for characterizing and evaluating reformulationtechniques for reasoning about physical systems. Sec-tion 3.1 introduces the components of the framework.Section 3.2 shows how selection of a sequence of refor-mulations can be reduced to a planning task. Finally,Section 3.3 introduces the attributes necessary to char-acterize a reformulation technique, so that selectioncan be performed.
3.1 Components of the framework
Reformulation is a mapping of an original problem intoa new problem, as shown in Fig. 2.
We distinguish two primary components, the prob-lem and the reformulation, and a composite compo-nent, the strategy, obtained from composing the formertwo.
3.1.1 Problem
We define a problem as a three-tuple: Problem =(Query, Form, Aaamptn). Query specifies the questionthat the user is trying to answer. Form denotes the for-mulation, i.e. the conceptualization of the domain. Fi-nally, kasmptn designates the conditions under whichthe formulation is valid, e.g. the domain of applicabil-ity and the temporal granularity. In Fig. 3, the prob-lem P1 is represented as a node.
®Figure 3: Problem, P1.
3.1.2 Reformulation
A reformulation technique is applied to an originalproblem Problem1 = (Query1, Forint, Assmptn1), toproduce the reformulated problem, Problem2. Wedescribe the reformulation technique as a tuple:Reformulation ---- (Cond, Proc). Cond denotes the ap-plicability conditions and Proc denotes the procedurethat maps the original problem into a reformulatedone. Cond is a set of conditions that must be satis-fied by Problem1 for the reformulation method to betechnically applicable. It must be noted that Cond is anecessary condition for the applicability of Proc. Procis a computable procedure that realizes a mapping. InFig. 4, the reformulation Ro is illustrated as a tran-sition between nodes representing two problems P1 toP2.
Figure 4: Applying reformulation 1~ to an initial problemto produce a reformulated problem.
As mentioned in the introduction, the decision toperform a reformulation may be motivated by theavailability of a suitable problem-solving engine and its
Choueiry 23
performance for solving a problem2. A solution engineis applied to a problem to produce a result as an an-swer to the query. We use the term "engine" broadlyto include anything from an algorithm, to a special-purpose simulation program, to a general-purpose so-lution pacl~ge such as Mathematica®. There could bemultiple engines at one’s disposal to solve the originalor reformulated problems. Alternately, there may benone, when the problem is too difficult.
It follows from our definition of reformulation thata solution engine is nothing but a reformulation thatpartially or completely answers a query. More specif-ically, the information necessary to answer the queryexists implicitly in the problem encoding (i.e., formu-lation, query, and assumptions). A solution enginemerely manipulates the problem to make this implicitanswer explicit. As a consequence, all subsequent dis-cussion of the general notion of a reformulation proce-dure also pertains to what has traditionally been calleda problem-solving engine or solution engine.
3.1.3 Strategy
A reformulation is simply a mapping from one problemencoding to another. Thus, a reformulation can be un-derstood as a step towards providing an answer to thequery. A sequence of reformulations, which may in-clude one or more engines, constitutes one strategy foraddressing a task. Execution of a strategy constitutesproblem solving.
Figure 5: Strategy.
We define a strategy Si, denoted Si =[P1Ra...RzPi], to be a sequence of reformula-tions, [Ra...Rx] that is applied to an originalproblem P1- The path [P1RaP2... RxPi] in Fig. 5 isan example of such a strategy. Any subsequence, St,,of Si, starting at P1 and stopping at any intermediaryproblem Pk, between P1 and Pi, is also a strategy,and is called a sub-strategy of Si.
3.2 Reasoning as plan execution
We perceive reasoning about physical systems to pro-ceed according to processes identified in Fig. 1. Ac-cording to this figure, the content of the initial input,i.e. the task description, is gradually modified by acombination of any number of processes culminatingin an answer to the query. Given our definition ofreformulation, any of these processes is a reformula-tion. The stage of processing distinguishes whether
2In this paper, we do not address reformulations thatapply to the engine itself, as proposed in [8], because suchreformulations do not seem to arise in the class of problemswe address.
24 QR-98
the reformulation is applied to a collection of modelfragments, an equational model, or to one or more so-lutions. Reasoning about physical systems is thus asuccessive application of reformulation procedures thattransforms an initial problem encoding.
Problem solving involves the successive applicationof reformulation procedures to an initial problem en-coding to produce a final problem encoding. Clearlythere may be multiple sequences of reformulations thatmay be applied to address the problem solving task,as illustrated in the figure below. Identifying such se-quences of reformulation procedures can be viewed asa planning problem in which the states are problem en-codings, the transition (or actions) are reformulations,and the plans are strategies. Hence, the task of rea-soning about physical systems becomes an executionof the selected plan.
R ....... L:~
Figure 6: Reasoning about physical systems is a plan exe-cution.
In Fig, 6, we illustrate a tree of four alternativestrategies. In practice, as for planning, resources maybe limited, and one may want to associate a utility orobjective function to the problem solving task. We ex-pect the user to provide the goal of the problem-solvingtask in terms of a goal test and of an objective functionthat specifies the importance of some desired featuresof the problem and the resources available. In this con-text, selecting an optimal plan or strategy becomes amulti-criteria optimization problem.
3.3 Evaluating and comparingcomponents
To articulate the goal driving this planning process, weidentify features of a problem, of a reformulation, andof the application of a reformulation to a problem thatare relevant to selection of reformulation procedures.These features are divided into sets, relative to thecomponents of our framework. The sets of features areincrementally augmented and refined as one explores,defines, and proposes new reformulation procedures.There are three main categories of sets, depending onwhether they evaluate a component of the framework(called evaluators, denoted Evals), assess the changedue to a reformulation (called change indicators, de-noted D iff s), or compare components obtained by dis-tinct strategies (called comparators, denoted Compars).We first introduce these sets, then we discuss each of
them in further details for each of the components towhich they apply.
Evaluators. For comparing the reformulation tech-niques described in the literature, and for assessingand comparing reformulation techniques in the con-text of problem solving, we must first define mea-sures or evaluators that designate relevant charac-teristics of problem and the reformulation or theirinter-relationships. We distinguish three sets of suchevaluators, Evalsprob, Evalsr.for~, and Evalsstratcorresponding to evaluators that assess aspects of aproblem, a reformulation technique, and a strategyrespectively.
Change indicators. For tracking the evolution ofa problem-solving task along successive reformula-tions, it is important to be able to characterize thechanges that occur in the problem as the result ofreformulation. One way to capture these changesis to measure the difference in the values returnedby the evaluators before and after one or more re-formulations. Another way is to measure changesbetween the application of different strategies to thesame initial problem encoding. The values returnedby change indicators are not necessarily quantita-tive; they could be qualitative or logical, but mustat least capture some notion of change or evolution.We identify two sets of change indicators, DiffSprob,and Dif fSstrat.
Comparators. An essential aspect of our frameworkis the ability to articulate the relative merits of alter-native strategies for problem-solving by comparingbetween their components. We introduce two sets ofcomparators, ComparSprob, and Comparsstrat.
3.3.1 ProblemBelow we introduce the terminology for characterizingand comparing problems. We illustrate this vocabularyin Fig. 7.
1. Evalsprob(Pi) denotes the set of evaluators for as-sessing some aspects of a problem Pi, including thequality of the answer ’contained’ in Pi.In the most general case, the elements in this setcan be defined with respect to any of the three el-ements of the problem, i.e. the formulation, query,or assumptions. In examining a wide collection ofreformulation techniques, we have found that thequery and assumptions often remain unchanged be-fore and after reformulation, and that most evalua-tors are functions applied exclusively to the formula-tion. Counterexamples do exist and will be discussedin a forthcoming technical report.These evaluators usually provide a quantitative as-sessment of some aspect of the formulation (e.g.,size) or of its logical properties (e.g., provability andrefutation). They can also address qualitative, lessquantifiable, aspects of the formulation (e.g., expres-sive power). For a system of equations, an example
.
......... comp, b
¯ ’. ..."
EvalSprob D|ffSprob
Figure 7: Evaluating and comparing problems.
of a quantitative evaluator is the number of equa-tions or variables, or the number of terms per equa-tion; an example of a qualitative evaluator is adher-ence of the equations to some canonical form. Otherevaluators of the formulation that appear in the lit-erature include scope [22] (which is the range of phe-nomena that it can describe), expressiveness, syntac-tic form, simplicity, generality, relevance, absence ofirrelevant information, and language restriction tofamiliar terms. It is important to define an evalu-ator in sufficient detail. In the case of simplicity,for example, we must define the specifics of how itis measured (e.g., the number of variables/equationsin a equation set, or the number of components in amodel).Some of the evaluators in Evalsprob are dedicatedto assessing the result to the query as it is madeexplicit in Form. Examples of such evaluators arethe soundness of the result, and its precision. Theseare typically the evaluators to use in the test thatdetermines whether the goal of the planning processis achieved.
DiffSprob(Pi, Pj) denotes a set of effects of the refor-mulation, thus measuring a change in some featureof the problem. Any element in this set measuresthe change between the corresponding elements inEvalsprob(Pi) and Evalsprob(Pj), such that Pi andPj are situated along the same strategy Sk. WhenPi and Pj are adjacent in Sk, DiffSprob(Pi, Pj) in-dicates the effects of applying a reformulation to Pi.When Pi and Pj are not adjacent in Sk, it indicatesthe effects of the application of a sequence of refor-mulations.One possible effect of reformulation on the prob-lem is to improve cognitive insight; this is com-mon at the solution reformulation stage, see Fig. 1.For example, if the original formulation is too com-plex for a user to understand, reformulation mayproduce a description better suited to human un-derstanding. Other examples of effects on theformulation are the following properties theoremincreasing/decreasing/constant, upward/downwardsolution [21], upward/downward-failure [23], orderedmonotonicity [10], and safety [2].Similarly to the case of EvalSprob (Pi), some elementsof DiffSp~ob(P~,Pj) are dedicated to assessing thechange of some features of the result. An example
Choueiry 25
of such an effect is a 10% loss in the accuracy of theresult.The change between two problems, Pi and Pj, re-flects the effect of a reformulation (alternatively, sequence of reformulations) on Pi. This outcome canalso be predicted from considering the mathematicalproperties of the reformulation itself when applied toPi (alternatively, the composition of the propertiesof the sequence of reformulations). There are, thus,two redundant ways of expressing this change, ei-ther as Diff%rob(Pi, Pj) or as Effect%rob(Rk, P~),which simply captures the effects of applying the re-formulation Rk to Pi. For instance, Struss [20] con-siders reformulations procedures, called representa-tional transformations, that are surjective, and notinjective; Giunchiglia and Walsh [8] study proce-dures that are computable surjective total functionsbetween two formal systems. The relationship be-tween Effectsprol~, Evalsprob and Dill%rob, can bedescribed as follows:
where f and g are functions to be defined. Forinstance, one may want to state that a reformu-lation Rk doubles the size of P~, thus specifyingEffeCtSprob(Rk,Pi), or that the size of Pj is twicethat of P~ by taking the ratio of the elements ’size’in Evalsprob(Pj) and Eval%rob(Pi), referring to an element of Diffsp~ob(Pi, Pj). We choose include both representations and to not arbitrarilyfavor one possibility over the other.
3. Comparsp~ob(Pi,Pj) denotes a set of effects of twodistinct strategies St~ and St applied to a given prob-lem by measuring a change in some features betweenPi and Pj resulting from applying Sk and St to theproblem. It measures the relative merits, with re-spect to the problem, of two alternative reasoningstrategies.
Similarly to Eval%rob and DiffSprob, Comparsprobencompass elements dedicated to comparing somefeatures of the results in the problems obtained bythe two alternative strategies.
3.3.2 Reformulation
For the reformulation, we introduce Evals~,f a set ofevaluators for assessing the reformulation, i.e. the con-ditions and the procedure.
This set is somehow complex. It contains evalua-tots that describe the reformulation technique in abso-lute terms (e.g., the size of the code, the programminglanguage it is written in, the price of a commercialsoftware, the human effort required to exploit it, andperhaps whether it requires a special hardware or ahuman expert).
Moreover, Evals~o~ includes functions that assessthe behavior of the reformulation technique relativeto a given problem, for instance, its time complexity
when applied to the problem. An example of an eval-uator that applies to the conditions of the reformula-tion, Cond, is the tractability of verifying them. A typi-cal and important evaluator is the computational com-plexity of the procedure, Proc, with respect to the orig-inal problem, denoted Complexity(Proc, Probleml).
3.3.3 Strategy
Below we introduce the evaluators, change indicatorsand comparators applicable to strategies, while illus-trating them in Fig. 8.
E TalSstrat
Compm’Sstrat
.̄.~__ ~__’t~_ ~__~__.,..’-a~;
II~tra¢
Figure 8: Evaluating and comparing strategies.
I. Evals,t~,t(S~) denotes a set of evaluators for assess-ing some aspects of a problem-solving strategy Si.An element in this set is obtained by considering,over a given path, some combination of the values ofan element Evals~o~, which measure features of thereformulation technique as introduced above. En ex-ample of such an evaluator is the cost of the strat-egy, assessed as the sum of the costs of applyingthe reformulations to the corresponding problem andthe absolute cost of the procedures (e.g., commercialprice).These are typically the evaluators to use in the ob-jective function that expresses the preferences andmanages the resources of the planning process.
2. Diffs,trat(Si,Sj) denotes a set of the effects of ex-tending a strategy Si by one or more reformulationsteps into a strategy Sj. Any element in this setmeasures the change between the corresponding el-ements in Evals,tr,t(S~) and Evals,trat(Sj), that Si is a sub-strategy of Sj. Examples of suchelements are increase in cost, loss of time, and con-sumption of available resources.
3. Compars,trat(Si, Sj) denotes the set of effects of twodistinct reasoning strategies on an original prob-lem by measuring a change of some feature inEvals,t~t(Si) and Evals,t~t(Sj), thus yielding assessment of the relative merits of the strategies.An element in this set is obtained by measuring thedifference, or ratio, of an element of Evals,trat foreach path.Traditionally, a reformulation is said to be cost-wisebeneficial when the cost of reformulating the prob-lem and that of solving the reformulated problem do
26 QR-98
not exceed the cost of solving the original problem.This is typically an element of Comparsstrat.Since one of the goals of reformulation is to improveoverall problem-solving performance, the reformula-tion procedure itself should not significantly add tothe computational cost. However, sometimes thereis no solution engine applicable to the original prob-lem, and consequently any amount of effort to re-formulate to make it solvable is justifiable. A cost-intensive reformulation may also be justified whenit is performed off-line to improve runtime perfor-mance of a system.
3.3.4 RemarkObserve that the evaluators for the problem, refor-mulation, and strategy are not necessarily indepen-dent. For example, simplification of a set of equa-tions often reduces the size of the formulation (mea-sured by Eval~rob) and reduces the cost of the refor-mulation (measured by Evalare~ and consequently byEvals.trat), at the expense of also reducing the preci-sion of the result (measured again by Evalsprob).
4 Illustrative examples
In this section, we examine three reformulation tech-niques described in the literature from the perspectiveof our framework for reformulation. These techniquesare representative of the types of reformulation thatcan occur at the three stages of reasoning about phys-ical systems. Each of the reviewed examples consistsof a summary of the reformulation technique and thedesiderata, followed by a characterization of the refor-mulation procedure in terms of our framework. Dueto space limitations, effects are mostly summarized inprose rather than exact and detailed definitions of rel-evant evaluators. Moreover, the authors do not pro-vide a comparison of their techniques with other pro-cedures, possibly because none exists. Thus, we willnot discuss comparators of problems and strategies inthe context of these examples.
4.1 Model reformulation: Simplification
Nayak and Joskowicz [16] propose a model reformula-tion technique that simplifies a compositional modelof a device, while maintaining its ability to providea causal explanation of the expected behavior of thedevice.
The primary objective of their work is to performefficient compositional modeling for generating parsi-monious causal explanations of the functioning of adevice. They provide tools for model-fragment libraryindexing and selection to support the construction ofdevice models. The reformulation procedure is appliedto the model thus built in order to simplify it. Wefocus here on this simplification process.
Given a device description, the expected behavior ofa device, and the above mentioned tools, the authorsprovide a model building algorithm that composes an
initial adequate model of the device. This initial modelis adequate in that it explains the expected behavior,includes significant phenomena, and excludes insignif-icant and irrelevant phenomena. However, it may notbe as parsimonious as it could be; that is, it may bepossible to further approximate the model fragments,according to the causal approximations defined in thelibrary, while maintaining the structural and behav-ioral constraints of the device, and the ability of themodel to explain the expected behavior. The refor-mulation procedure transforms the composed adequatemodel into one that is both adequate and parsimo-nious. This procedure is predicated on the fact thatall model fragment approximations provided in the li-brary are causal and acyclic.
The reformulation procedure is portrayed in ourframework as follows. Generating causal explanationsis the stated problem-solving objective, but the au-thors do not propose a specific problem solver. As aconsequence, we do not evaluate this model reformu-lation in the context of a larger strategy that includesthe generation of causal explanations, and we restrictour evaluation to only one transition, corresponding tothe model simplification. As a result, we will not dis-cuss evaluators that apply to the strategy, but only tothe problem and reformulation. Nevertheless, the au-thors observe that their notion of simplicity "does notguarantee that a simpler model will be more efficientto simulate or will produce simpler causal explanationsthan more complex ones." Thus, their choice for sim-plicity strongly affects the quality of the explanationgenerated by the overall strategy. The metric for eval-uating the reformulation procedure is parsimony of theproblem formulation.
Characteristics of the problem:
Form1 comprises:
¯ an adequate model of a device that consists of or-dinary differential equations, algebraic equationsand qualitative equations,
¯ the expected behavior of the device,
¯ structural and behavioral constraints, and¯ a library of model fragment approximations, in
which approximations are causal and the approx-imation relation is acyclic.
Form2: A parsimonious adequate model of a deviceand the above-mentioned library.
hssmptn1 = hssmptn2: The equations are restrictedto time-varying and equilibrium lumped parametermodels.
Characteristics of the reformulation:
Choueiry 27
Proc: Input to the reformulation procedure is an ade-quate compositional model that may contain unnec-essary model fragments and may not be as approxi-mated as possible.The reformulation procedure exploits two operators:(1) Replacement of a model fragment by one of itsimmediate approximations, as defined in the modelfragment library; and (2) Removal of an unneces-sary model fragment. The first operator is appliedrepeatedly ensuring that the resultant model can ex-plain the expected behavior. This is achieved by anorder of magnitude reasoner. The second operator isthen applied, again ensuring that the expected be-havior can be explained and that all the structuraland behavioral coherence constraints are satisfied.Note that the reformulation procedure generates onesimplest adequate model. More than one may existbut the procedure stops after finding the first.
Con& The approximations in the model fragment li-brary must be causal and the approximation relationacyclic.
Evaluators and effects:EVa.lBprob: The problem is evaluated with respect to
the parsimony of the problem formulation. The ef-fect of reformulation on the problem, E:~:fectsprob,is the simplest compositional device model that willexplain the expected behavior.
Evalsro~orm: The reformulation is evaluated withrespect to the complexity of the reformula-tion procedure relative to the problem encoding.With respect to this problem and reformulation,Complexity(Proc, Pt) is assessed to be tractable,provided the order of magnitude reasoning used toverify that the simplified model still explains the ex-pected behavior is approximated to be polynomial.The reason for the tractability of the procedure isas follows. Because of the compositional modelingparadigm and the provision of causal approxima-tions, the reformulation algorithm need not considerall combinations of model fragments during simpli-fication. It considers each model fragment indepen-dently, and replaces it by one of its immediate sim-plifications according to the causal approximationrelation. The algorithm computes the simplest ad-equate model (where simplest means that no modelfragment can be replaced by a simpler one that satis-fies the expected behavior), and it stops after findingthe first adequate simplest model.
4.2 Equation reformulation and solutionbuilding: linearization and stability
This example illustrates a problem-solving strategythat exploits linearization to determine the stability ofa particular class of nonlinear systems. The problem-solving strategy consists of two sequential reformu-lations - an equation reformulation Ra which maps
28 QR-98
problem encoding Pl into problem encoding P2, anda problem-solver reformulation Rb which maps P2 intoP3.
Linearization is a strategy commonly used to evalu-ate the stability of a nonlinear system near one of itsequilibrium points. The reformulation is used to facili-tate the inference of this stability property: it replacesthe analysis of the stability of the nonlinear system bythat of the linear system, derived by linearization of theequations of the nonlinear system. The stability of theresulting linear system is determined by the locationof the eigenvalues of the system matrix in the complexplane. The main rationale for exploiting this strat-egy is that "for small deviations from the equilibriumpoint, the performance of the system is approximatelygoverned by the linear terms. These terms dominateand thus determine stabihty-provided that the linearterms do not vanish" [13]. If this is not the case, aseparate analysis is required.
In general, no problem-solver reformulation is avail-able to directly determine the stability of nonlinearsystems3. Hence, the motivation for performing thissequence of reformulations is engine-driven problemre-encoding. The metric for evaluating the strategyis with respect to the ’answerability’ of the stabilityquestion. The strategy is deemed to be successful if ananswer to the stability question can always be givenand a stability region near an equilibrium point esti-mated. The reformulation procedures are portrayed inour framework as follows.
Characteristics of the problem:Forint comprises:
¯ a set of nonlinear differential equations x(t) f(x(t)) of n variables,
¯ an equilibrium point for the nonlinear system.
Form2 is a set of time-invariant linear differential equa-tions of n variables that approximate Form1 at anequilibrium point.The system matrix of the linearized system is theJacobian of the nonlinear system computed at theequilibrium point. It has the same number of vari-ables and equations as Form1.
Form3 is Form2 plus the result, i.e. one symbol of(stable, unstable, unknown}.
Queryt = Query2 = Query3: Determine the stabilityproperties near an equilibrium point.
Assmptnt: The equations are time-invariant4.
3Unless one is given a Liapunov function that can beused to prove stability within a region containing the equi-librium point.
4The dynamic behavior of a continuous system of n vari-ables is described by a set of differential equations of thefollowing general form: ±(t) = f(x(t), t). The system is to be time-invariant when the functions f do not dependexplicitly on time, i.e. x(t) = f(x(t)).
Assmptn2 = hssmpl;n3: None.
Characteristics of the reformulations:
Proca comprises the following steps: (1) Computationof the Jacobian of the nonlinear system. (2) Evalu-ation of the Jacobian at the equilibrium point.
Conda: Input must conform to Form1, and the equa-tions must be time-invariant.
Procb determine the stability near a specific equilib-rium point, the eigenvalues, Ai, of the system matrixof the linearized system are determined. The stabil-ity of the original system is inferred from that of thelinearized version as follows:
. If at least one eigenvalue is found to bein the right-hand side of the complex plane(3i, Re(Ai) > 0), the nonlinear system is unstable.
¯ If all eigenvalues are in the left-hand side of thecomplex plane (Vi, Re(A~) < 0), the nonlinear tem is stable.
¯ If all eigenvalues are in the left-hand side of thecomplex plane, but at least one has a zero realvalue (3i, Re(Ai) = 0), then no conclusions be drawn for the stability of the nonlinear system,and one must analyze the higher order terms ofthe function f.
Condb: The problem must be formulated as describedin Form2.
Evaluators and effects:
Evalsp,ob(P2): Following equation reformulation, theproblem is evaluated with respect to the syntac-tic form of the problem formulation. The ef-fects of the equation reformulation on the problem,Effect~rob(P2), are that the set of equations is nowlinear. The system matrix of the linearized systemis the Jacobian of the nonlinear system computed atthe equilibrium point.
Evals~.~o~ffi(Ra): The reformulation is evaluated withrespect to the complexity of the reformula-tion procedure relative to the problem encoding,Complexity(Proc,, PI), which is O(n3).
Evalspro~(P3): Following solution building, the prob-lem is evaluated with respect to whether the stabil-ity of the system near an equilibrium point is es-tablished, refuted, or remains undetermined. Thiswill depend on the problem. This technique cannotspecify the boundaries of the stability region nearthe equilibrium point. Moreover, sometimes P3 isnot conclusive and one must analyze the effects ofneglected higher-order terms.
Evalsr,~or,(Rb): The solution building procedurecomputes the eigenvalues of a system matrixand it is evaluated with respect to complexity,Complexity(PrOcb, P2), which is O(n3).
4.3 Solution reformulation: Behaviorabstraction for explanation
In [14], Mallory et al. propose to summarize the re-suits of the qualitative simulation of a physical systemin order to help users recognize "basic patterns of be-havior." Their goal is to support human understandingof the solution space. This reformulation is a typicalinstance of solution reformulation.
Given the user’s query and the complete behaviortree of a simulation, generated by QSIM [11], the re-formulation procedure summarizes the behavior of thesystem by generating a behavior graph that retainsonly those aspects of the behavior tree relevant tothe query. The procedure examines the labels of thenodes in the original tree, discards irrelevant informa-tion from the labels, and merges adjacent nodes ac-cording to a well-defined strategy. The task of inducingpatterns of behaviors and producing a higher-level de-scription of the resulting graph is currently entrustedto the user, but the authors plan to extend their workin this direction.
The motivation here is to enhance cognitive insightinto the solution space, i.e. the formulation. The qual-ity of the formulation is measured by (1) the size of thebehavior graph and its tractability with respect to ma-nipulation and understanding by a human user; (2) user’s subjective opinion of the quality of the summaryprovided by the behavior graph; and (3) soundness andcompleteness of the behavior graph with respect to theoriginal behavior tree, defined as follows. Soundness:any reformulated behavior corresponds to at least oneoriginal behavior. Completeness: all original behav-iors are represented in the abstract graph. The refor-mulation procedure is portrayed in our framework asfollows.
Characteristics of the problem:Formi: A behavior tree representing the qualitative
simulation of the behavior of a physical system.Each node of the tree represents a qualitative stateof the system.
Form2: A graph whose nodes are either nodes or ag-gregates of two or more nodes of the original tree.
query1 = [~uery2: Summarize the behavior of a speci-fied subset of quantities/variables in terms of a spec-ified subset of their so-called "methods" (e.g., qual-itative values and direction of change).
Assmptni = Assmptn2: None.
Characteristics of reformulation:
Proc comprises the following steps: (1) Label eachnode in the tree with the values and methods of thespecified variables. (2) Generate a graph by aggre-gating nodes that have the same labels and satisfysome adjacency conditions. The authors provide adefinition for the adjacency of nodes that guarantees
Choueiry 29
that all significant behaviors in the original behaviortree are reproduced in the behavior graph.
Cond: Input must conform to Fermi.
Evaluators and effects:
EvalSprob: The problem is evaluated relative tothe solution formulation. As explained above,the following are evaluators for the problem:Size, Understandability, Soundness, andCompleteness.The following general observations can be made rel-ative to various differences and effects, Effectsp~oband DiffSprob on the evaluators. With respect toSize, the size of the abstracted-behavior graph issmaller than or equal to that of the original one.With respect to Soundness and Completeness, theabstracted behavior graph is guaranteed to keep onlythose states pertinent to the query, and the authorsprovide a proof of the soundness and correctness ofthe reformulation procedure. Finally, as an obser-vation with respect to Understandability, the au-thors report that the user may have to experimentwith different specifications of the query in order toachieve a satisfactory summary of the behavior ofinterest.
Evals~o~or=: The reformulation is evaluated with re-spect to the complexity of the solution reformula-tion procedure relative to the problem encoding.With respect to this problem and reformulation,Complexity(Proc, Px) is polynomial.
5 Related workVarious theories of reformulation including abstractionand approximation have been proposed in the litera-ture. Some of these theories provide an encompassinghigh-level characterization. Others restrict their scopeto some specific aspect (e.g., cost or faithfulness of re-sults). These theories proved to be essential to ourunderstanding of reformulation, but we found them tobe of limited practical use in automating the selectionand application of reformulation techniques.
Giunchiglia and Walsh [8] introduced a general the-ory of abstraction. They introduced a general charac-terization of reformulation and its properties. BothCremonini et al. [2] and Nayak and Levy [17] ex-plored abstraction theories that are restricted to logicalsystems and to abstraction techniques that preserveconsistency and correctness of proofs. None of thesetheories make extensive analysis of complexity issues,nor do they provide the terminology for quantitativelyevaluating the effects of reformulation. In contrast,the body of research on approximations in the compu-tational complexity community [18], provides rigorousevaluation criteria with respect to cost while neglectingto address issues of expressiveness of representations,which are fundamental in artificial intelligence.
30 QR-98
In [23], Weld and Addanki take a task-driven ap-proach to reformulation and adopt Tenenberg’s vocab-ulary [21] for describing the effects of the reformula-tion on the formulation, only. In [3], Davis studies ap-proximation and abstraction and focuses on the prac-tical application of reformulation techniques appliedto reasoning about solid object kinematics. Davis toostresses that the selection of the reformulation tech-nique must be task-driven and in order to satisfy somewell-defined criteria. Neither works, however, providesa general framework for reformulation, or identifies at-tributes for describing and evaluating reformulationtechniques.
As a final note, multiple perspectives are commonlysought in automated reasoning to improve performanceof the reasoning. We view generation of such perspec-tives as a reformulation only when the mapping fromone perspective to another is well articulated.
6 ConclusionsIn this paper we provide a practical framework forcharacterizing, evaluating, and selecting reformulationtechniques, with the long-term goal of automatingtheir selection and application in the context of rea-soning about physical systems. While the focus of ourresearch has been on reasoning about physical systems,and hence all our examples are drawn from this do-main, the framework developed appears to be applica-ble to a broad range of tasks and domains.
We identify the three stages of reasoning about phys-ical systems at which interesting reformulations maybe performed. However, we do not require that thereasoning transition through all stages, or that it do sosequentially. Our study uncovered two simple, not yetarticulated, observations. First, solving engines cannaturally be cast as reformulations, eliminating theimplicit distinction between reformulations and solv-ing engines. Second, the task of selecting a sequenceof reformulations to achieve the goal of problem solvingcan be cast as a planning problem. Hence, reasoningabout physical systems can be reduced to execution ofthe selected plan.
Our framework provides the terminology to specifyand assess the three components in a complex reformu-lation process (namely, the problem, the reformulation,and the strategy) by providing evaluators of the prop-erties of these components, and comparators of theirrelative merits. We believe that our framework is suf-ficiently flexible to accommodate previously proposedproperties and metrics for reformulation. We have alsocollected a variety of evaluators reported in the litera-ture, and structured them according to our framework.The evaluators discussed here are not intended to beexhaustive and will certainly need to be augmentedwhen the framework is extended to comprise reformu-lation of solution engines or to apply in other types ofproblem domains.
In an effort to evaluate this framework, we have used
it to characterize numerous reformulation techniques,three of which we have presented in this paper. Thisprocess was not straightforward, especially since wewere initially introducing a conceptual distinction be-tween reformulations and solving engines. The cur-rent framework is the result of countless iterations overthe analysis of the examples, and our perception ofproblem-solving goals and strategies. Further evalua-tions still need to be carried out, which we intend toreport in a forthcoming technical report.
There are numerous avenues for future work: (1) ex-tend our framework to encompass engine reformula-tion; (2) assess the usefulness of our framework fortasks other than query answering (e.g., design) anddisciplines other than automated reasoning (e.g., cog-nitive modeling); and (3) study, in more detail more formally, the composition and inverse mappingof reformulations.
AcknowledgmentsThe authors are grateful to William Buchanan for var-ious discussions, to Lee S. Brownston for proofread-ing an early version of this document, and to AAAIanonymous reviewers for constructive comments. B.Y. Choueiry is supported by a fellowship for advancedresearchers from the Swiss National Science Founda-tion. S. McIlraith was supported by the Natural Sci-ences and Engineering Research Council of Canada(NSERC) and by Xerox Palo Alto Research Center.
References[1] Daniel J. Clancy and Benjamin Kuipers. Static and
Dynamic Abstraction Solves the Problem of Chatterin Qualitative S!mulation. In Proc. of AAAI-97, pages118-125, Providence, Rhode Island, 1997.
[2] Roberto Cremonini, Kim Marriott, and Harald Sen-dergaard. A General Theory of Abstraction. In Pro-ceedings of the 4th Australian Joint Conference on Ar-tificial Intelligence, pages 121-134, Australia, 1990.
[3] Ernest Davis. Approximation and Abstraction in SolidObject Kinematics. Technical Report TR706, NewYork University, New York, NY, 1995.
[4] Brian Falkenhainer and Kenneth D. Forbus. Composi-tional Modeling: Finding the Right Model for the Job.Artificial Intelligence, 51:95-143, 1991.
[5] Adam Farquhar. A Qualitative Physics Compiler.In Proc. of AAAI-94, pages 1168-1174, Seattle, WA,1994.
[6] Kenneth D. Forbus. Qualitatite Process Theory. Ar-tificial Intelligence, 24:85-168, 1984.
[7] Kenneth D. Forbus. The Qualitative Process Engine.In Daniel S. Weld and Johan de Kleer, editors, Read-ings in Qualitative Reasoning about Physical Systems,pages 220-235. Morgan Kanfmann, San Mateo, CA,1990.
[8] Fausto Giunchiglia and Toby Walsh. A Theory of Ab-straction. Artificial Intelligence, 57:323-389, 1992.
[9] Tom R. Gruber and Pierre O. Gautier. Machine-generated Explanations of Engineering Models: aCompositional Modeling Approach. In Proc. of the13 ~h IJCAI, pages 1502-1508, Chamb6ry, France,1993.
[10] Craig A. Knoblock, Josh D. Tenenberg, and QiangYang. Characterizing Abstraction Hierarchies forPlanning. In Proc. of AAAI-91, pages 692-697, Ana-heim, CA, 1991.
[11] Benjamin Kuipers. Qualitative Simulation. ArtificialIntelligence, 29:289--338, 1986.
[12] Alon Y. Levy, Yumi Iwasaki, and Richard Fikes. Au-tomated Model Selection for Simulation Based on Rel-evance Reasoning. Artificial Intelligence, 96:351-394,1997.
[13] David G. Luenberger. Introduction to Dynamic Sys-tems : theory, models, and applications, chapter Anal-ysis of Nonlinear Systems. Wiley, New York, 1979.
[14l Richard S. Mallory, Bruce W. Porter, and Benjamin J.Kuipers. Comprehending Complex Behavior Graphsthrough Abstractions. In Tenth International Work-shop on Qualitative Physics. AAAI Technical ReportWS-96-01, pages 137-146, Fallen Leaf Lake, CA, 1996.
[15] P. Pandurang Nayak. Causal approximations. Artifi-cial Intelligence, 70:27"/-334, 1994.
[16] P. Pandurang Nayak and Leo Joskowicz. EfficientCompositional Modeling for Generating Causal Expla-nations. Artificial Intelligence, 83:193-227, 1996.
[17] P. Pandurang Nayak and Alon Y. Levy. A SemanticTheory of Abstractions. In Proc. of the 14 th IJCALpages 196-203, Montreal, Canada, 1995.
[18] Christos H. Papadimitriou. Computational Complex-ity, chapter Approximation and Complexity, pages299-328. Addison-Wesley Publishing Company, Read-ing, Massachusetts, 1994.
[19] Jeff Rickel and Bruce Porter. Automated Modeling forAnswering Prediction Questions: Selecting the TimeScale and System Boundary. In Proc. of AAAI-9.~,pages 1191-1198, Seattle, WA, 1994.
[20] Peter Struss. On Temporal Abstraction in Qualita-tive Reasoning (A Preliminary report). In Proceedingsof the Seventh International Workshop on QualitativeReasoning about Physical Systems, pages 219-227, Or-cas Island, Wa, 1993.
[21] Josh D. Tenenberg. Inheritance in Automated Plan-ning. In First International Conference on Knowl-edge Representation and Reasoning, pages 475-485,Toronto, Canada, 1989.
[22] Daniel S. Weld. Reasoning about Model Accuracy.Artificial Intelligence, 56:255-300, 1992.
[23] Daniel S. Weld and Sanjaya Addanki. Task-DrivenModel Abstraction. In 4th International Workshopon Qualitative Physics, pages 16-30, Lugano, Switzer-land, 1990.
