GOEL, Vinod and PIROLLI, Peter. the Structure of Design Problem Spaces

8/18/2019 GOEL, Vinod and PIROLLI, Peter. the Structure of Design Problem Spaces

1/35

COGNITIVE SCIENCE16, 395-429 (1992)

The Structure ofDesignProblemSpaces

VINOD GOEL AND PETER PIROLLIUniversity of California, Berkeley

It is proposed that there are important generalizations about problem solving indesig n activity that reach across specific disciplines . A framework for the study of

desig n is presented that (a) characterizes desig n as a rad ial category and fleshes

out the task environ men t of the prototypical cases: (b) takes the task environ-

me nt seriously: (c) shows that this task env ironme nt occurs in desig n tasks, but

does not occur in every non des ign task: (d) explicates the impa ct of this task

environment on the design problem space: and (e) demonstrates that, given the

structure of the information-proce ssing system, the features note d in the prob-

lem spaces of design tasks will not all occur in problem spaces where the task

environment is vastly different. This analysis leads to the claim that there are a

set of invariant features in the problem spaces of design situations that collec-

tively constitute a design problem space. Protoc ol studies are reported in whichthe problem spaces of three design tasks in architecture, mech anical engineer-

ing, and instructional design are explored ond compared with several protocols

from nondes ign problem-solving tasks.

The proper study of mankind is the science of design. (Simon, 1981, p. 159)

1. INTRODUCTION

Design s a quintessential ognitive ask. The activity of design nvolves hemental formulation of future statesof affairs. The productsof designac-tivity are external epresentations f suchpossible utures. Creative designis also one of the recognizable eatures hat distinguishes modern humansfrom other intelligent makers of artifacts (Mellars, 1989;White, 1989;

This a rticle is a condensed version of cha pters 3 and 4 of Gael’s (1991) dissertation. Fund-ing for this research wa s provided by the Office of Na val Res earch Cognitive Science Program,grant number NOOO 14-88-K-0233 to Peter Pirolli, and a My rtle L. Judkins Mem orial Fellowship, a Canada Mortgage and Housing Corporation Scholarship, a Gale Fellowship, and aresearch internship at the Sys tem Sciences Lab a t Xerox PAR C and CSLI at Stanford Univer-sity to Vinod Goel. We would like to thank Dan Berger, Kate Bielacxyc, Susan Newman, MiiReek er, and Mike Sipusic for com m ents and discussions throughout this project.

Correspondence and requests for reprints should be sent to Vmod Goel, Cognitive Neuro-

science Section, NIH/NIN DS /MN B, 9000 Rockville Pike, Bldg. 10, Room 5S209, Bethesda,MD 20892.

395


2/35

396 GOEL AND PIROLLI

Wynn, 1979,1981).Design s, therefore, undamentally mental, representa-tional, and a signature of human ntelligence:Features hat surely make t

an important subject of study n cognitivescience.The study of design n cognitive science s also timely in the context ofpast and recent developmentsn the field. Much of the earlywork on prob-blem solving (e.g., Newell & Simon, 1972) xaminedperformance n well-structured, emanticallympoverishedasks-such as puzzle olving-havingwell-definedgoals,problem states, and operators. ll-structured problems,whichhave ll-definedgoals,states, r operators, ave been eceiving ncreas-ing attention. Such research ncludesstudies of softwaredesign Guindon,

1989;Jeffries, Turner, Polson, & Atwood, 1981;Kant, 1985),mechanicalengineeringUllman, Dietterich, & Stauffer, 1988), nd the formulation ofadministrativepolicy (Voss,Greene,Post, & Penner, 1983). n addition,studiesof problemsolving n semantically ich domains nvolving the use ofmental models, such as physics (Gentner& Gentner,1983),have also n-creased ver the past decade.Design,of the sort we are nterested n, is ill-structured n that the tasks nvolve underspecified oals and operators. hekinds of knowledge hat may enter nto a design solution are practically

limitless. Furthermore,because esign nherently consists of the formula-tion of modelsof possiblestates of affairs in the world, it intrinsically in-volvesmental models and a rich set of semantics. Given such rends n thestudy of problemsolving,design s a compelling est bed or further ntegra-tion and advancement f theories of cognition.

There are also practical reasons or being interested n the study ofdesign.Designactivity hasmajor impact on constructionand manufactur-ing costsand capabilities.Although it is the case hat design accounts or

only 3-7% of product costs, and that the real money s spent during themanufacturingor constructionstage, t is also he case hat 80% of manu-facturing costs are committed during the first 20% of the design process(Dixon & Duffey, 1990).Analystshavenoted serious hortcomings n botheducationand practice of designand have ecently stressed hat a lack of abasic scientific understanding f the design process s endangering meri-can ndustry and productivity (Dertouzos, Lester, & Solow, 1989).

In this article, wepresent framework, theory, and method or the study

of design asks, along with empirical analyses of expert design n threediversedisciplines.Our goal s to address he knowledge nd cognitivepro-cesses f individual designers, lthough clearly, design s often the coor-dinated effort of many individuals (Curtis, Krasner, & Iscoe, 1988).Theframework is intended to encompass heories or models that generalizeacrossdesign asks, provide accounts of individual acts of design,and af-ford computationalaccountsof moment to moment information process-ing. In thenext sectionwe embedour work in the iteraturewith a verybrief

outline of past design esearch and then present our research rameworkand empirical analyses n subsequent ections.


3/35

GENERIC DESIGN 397

2. AN OVERVIEW OF DESIGN RESEARCH

The development of systematic theories of design has a long history withinthe design professions, dating back to Leon Battista Alberti’s (1450/1988)treatise on architecture. More recently, cross-disciplinary systematization ofdesign was attempted in the design methodology movement. In part, thismovement was spurred on by the large military and civilian projects of the1950s and 196Os, such as the development of the Polaris Missile and themoon landing. The design methodology movement responded with a numberof prescriptive proposals for the systematization of the design process. Anumber of researchers (Alexander, 1964; Archer, 1969) observed that designhad two components: a logical element and a creative element. Both werenecessary, but required very different abilities. The basic idea of the designmethodology movement was to develop systematic external methods andtools to carry out the logical analysis better, and to unburden the designer toengage in the creative aspects of problem solving (Cross, 1984).

One of the intellectual outcomes of this research was a consensus amongmany practitioners that, whereas the various design professions, for exam-ple, architecture, engineering, industrial design, urban design, and so on,differed in particulars, they nonetheless shared a common core that united-them as design professions and differentiated them from nondesign profes-sions such as medicine. This unifying core was thought to reside in the factthat design problem-solving activity involved the following sequence ofsteps:

1. An exploration and decomposition of the problem (i.e., analysis).2. An identification of the interconnections among the components.

3. The solution of the subproblems in isolation.4. The combination (taking into account the interconnections) of the par-

tial solutions into the problem solution (i.e., synthesis).

On this basis many researchers concluded that “the logical nature of the actof designing is largely independent of the character of the thing designed”(Archer, 1969, p. 76). These prescriptive proposals for revamping designmethods gave way to studies involving either the logical analysis of design

problems (Alexander & Poyner, 1966; March, 1976; Reitman, 1964; Rittel& Webber, 1973; Simon, 1973a) or empirical studies (Akin, 1979, 1986;Darke, 1979; Eastman, 1969) of design as it naturally occurs in the world(or at least the laboratory).

Another line of attack on design problems emanated from information-processing analyses of problem solving. Design problems that require somecreativity for their solution were identified as ill-defined or ill-structuredproblems by Reitman (1964). In other words, such problems initially have

underspecified or ambiguous specifications of their start state, goal state,or the function that transforms the start state to the goal state. Simon


4/35


(1973a) argued that there was nothing intrinsic about a problem, per se, thatmake it ill- or well-structured, but rather, that such properties could only be

determined by examining the relationship between the problem solver, itsavailable knowledge, and the problem to be solved. Furthermore, Simonconcluded that information-processing accounts were adequate to deal withboth ill-structured and well-structured problem solving.

Many recent analyses of design have been carried out in the frameworkof cognitive psychology or artificial intelligence (Akin, 1979, 1986; Brown& Chandrasekaran, 1989; Guindon, 1989; Jeffries et al., 1981; Kant, 1985;Mostow, 1985; Tong & Franklin, 1989; Ullman et al., 1988). However, as

noted in Goel and Pirolli (1989), much of the cognitive science research ondesign problem solving suffers from the following difficulty: Either theresearch has tended to concentrate on the analysis of discipline-specificdesign domains and has shied away from cross-disciplinary generalizations;or the term “design” has been applied to an increasingly large set of activitiesthat begins to drain it of substance. Such unlikely tasks as learning (Perkins,1986), communication (Thomas, 1978), letter writing, naming, and schedul-ing (Thomas & Carroll, 1979) have all been called design activities.

We differ from much of this research in believing neither that (a) designactivity characterizations must be discipline specific, n.or (b) that design is aubiquitous activity. There is no more reason to construe every cognitiveactivity as a design activity than there is reason to construe every cognitiveactivity as a game-playing activity or a natural language-generation activity.We assume that there are significant commonalities in the structure ofdesign problems and tasks across the various design disciplines, and thereare significant differences in the structure of design problems and non-

design problems. As such, we make a strong commitment to the study ofdesign as a subject matter in its own right, independent of specific tasks ordisciplines. We use the term generic design to refer to this study. We shouldnote, however, that we are not alone in thinking that there must be interest-ing generalizations to be drawn across broad classes of problems (e.g.,Chandrasekaran, 1983, 1987; Greeno, 1978).

Another way in which our work differs from other efforts in cognitivescience is that we explicitly acknowledge that our studies must emphasize

the analysis of design problems and the situations in which they occur. Al-though the importance of analyzing such tusk environments has been stressedin classical theories of human information processing and problem solving(Newell & Simon, 1972), such analyses are often taken for granted. In par-ticular, researchers dealing with design problems have not let such under-standing inform their data analysis and theorizing. We propose to take anddevelop the notion of design problems and the contexts in which they occurand let it seriously guide our cognitive characterizations.

Our research can therefore be viewed as an integration of two themes. Itis an attempt to show that design problem solving is a natural category of


5/35

GENERIC DESIGN 3954

activity and is interestingly different from nondesign problem-solving ac-tivity. This attempt draws on the analysis of design problems and situations,

and uses the results as both framing and explanatory constructs. In the nextsection we propose a framework in which to carry out such a study; subse-quent sections describe empirical analysis conducted within the framework.

3. A FRAMEWORK FOR STUDYING DESIGN

In order to study generic design, we need to develop criteria for demarcatingand recognizing design problems, and we need some understanding of theircommon characteristics. Our framework for such an analysis derives fromNewell and Simon’s (1972) information-processing theory of human problemsolving. Basically, a human problem solver is viewed as an information-pro-cessing system with a problem. A tusk environment is the external environ-ment, inclusive of the problem, in which the information-processing systemoperates. A problem space is a formalization of the structure of processing

molded by the characteristics of the information-processing system,. andmore importantly, the task environment. A problem space is defined interms of states of problem solving, operators that move the problem solvingfrom one state to another, and evaluation functions.

Our intuitions about generic design can be formulated in information-processing theory as an hypothesis about the design problem space.

Design Problem Space Hypothesis: Problem spaces exhibit major invariantsacross design problem-solving situations and major variants across design andnondesign problem-solving situations.

Our level of characterization of the design problem space will not be interms of states, operators, and evaluation functions, as is standard in manypsychological investigations (Newell & Simon, 1972), but will be stated in ahigher level vocabulary, in terms of a set of invariant features common to,or characteristic of, design problem spaces. Formally, this move to abstrac-tion is comparable to the development of process and data abstractions incomputer science (Liskov & Guttag, 1984).

The basic strategy for explicating the design problem space will be asfollows: (a) specify some salient features in the cognitive structure of thedesigner; (b) specify the salient common features or invariants in the taskenvironment of design problems; (c) show that they constitute a ratherunique set of invariants not found in just any arbitrary problem-solvingsituation; (d) let the problem space be shaped by these two sets of con-straints; (e) ‘note the structure of the resulting problem space and make “ex-planatory connections” between this structure and the invariants of the


6/35


design ask environment and the cognitive system; f) show hat the struc-ture of at least some nondesign roblem spaces s very different; (g) make

the Newelland Simon (1972) rgument hat, given he structure of the prob-lem solver as a constant across all cognitive activity, any interestingdif-ferences cross roblem spaces f vastly different taskswill bea function ofthe task environment; and Q on this basis, claim that these eatures are n-variants of design situations and collectively constitute a design problemspace. n the following subsections e specify he structure of the informa-tion-processing ystem, he structure of the design ask environment, andmake some predictionsabout the design problem space.

3.1 The Structure of the Information-Processing SystemIn the classicNewell and Simon (1972) heory of humanproblemsolving,acognitivesystem s a simple, relativelyunconstrained echanism. t is basic-ally a physicalsymbolmanipulationsystemwith memorystores short erm,long term, external),a processor, ensory eceptors, and motor effecters.There are, of course, severalmore sophisticated ccounts n the literature.We view our use of the Newell and Simon theory, as a way of providing a

reasonable irst-orderapproximationof human nformation processing hatis consistentwith a wide variety of more recentproposalsof human cogni-tive architectures Anderson, 1983; Newell, in press). The essential con-straints on human nformation processing roposed by Newell and Simonthat are relevant o our analysis-the limitations of short-term memory, ex-ternal memory, and the sequential nature of symbolic processing-wouldprobably show up in many theoretical alternatives.Even a connectionisttheory would probably incorporate similar assumptions o address he

human limitations in heeding nformation, and the sequential nature ofproblem-solvingactivity.

3.2 The Structure of the Design Task EnvironmentTask environments onsist of (a) a goal, (b) a problem, and (c) other rele-vant external actors (Newell & Simon, 1972). n many studiesof problemsolving, he emphasis as beenon how the structureand contentof apartic-ular problemgetsmappedonto the problem space. n contrast, our explica-

tion of the design ask environment nvolves ooking beyond he ndividualproblem and specifying he relevant external actors common o all designproblems.

Having put the problem hus, there are wo difficulties that must be dealtwith. First, beforeone can ook at the commonalities across he category ofdesignproblems,one must be able o specify what constitutes he category,or at least o identify membersof it. Second,having dentified he categoryin someway, one s confronted with the problem of identifying the aspects

of the task environment hat are relevant. Both of thesedifficulties have


7/35

GENERIC DESIGN 401

proven to be a notoriously challenging (Goel & Pirolli, 1989). We providecriteria motivated by categorization theory to address the first problem and

rely on intuitions, developed through immersion in the discipline of archi-tectural design,’ to generate criteria that address the second problem.It is our contention that design is too complex an activity to be specified

by necessary and sufficient conditions. Rather, design as a category exhibitswhat Rosch (1978) called “prototype effects.” Furthermore, it is whatLakoff (1987) called a “radical category,” a category in which there is acentral, ideal, or prototypical case and then some unpredictable butmotivated variations. Given this assumption, one could use a variety of con-

vergent operationalizations to determine the constituent structure of thecategory of design activity. For instance, if one shows people a list of profes-sions, for example, doctor, lawyer, architect, teacher, engineer, researcher,and asks which are the best examples of design professions, people willusually pick out the same few cases. In this list we believe the best exampleswould be architecture and engineering. We propose to call these central, orprototypical examples of design profession.

Having made this observation, we have a nonarbitrary and interesting

characterization of design activity. We are now in a position to take aserious look at the task environment of these prototypical design profes-sions and attempt to isolate some interesting common features. We note thefollowing overt features of design task environments (for an earlier approx-imation of this list, see Goel & Pirolli, 1989):

A. Distribution of information. As initially noted by Reitman (1964), thereis a lack of information in each of the three components of design prob-

lems. The start state is incompletely specified, the goal state is specifiedto an even lesser extent, and the transformation function from the startto goal states is completely unspecified.

B. Nature of constraints. The constraints on design task environments aregenerally of two types: (a) nomological, and (b) social, political, legal,economic, and so on. The latter consists of rules and conventions andare always negotiable. The former consists of natural laws and arenever negotiable. However, the constraints of natural law vastly under-

determine design solutions. Design constraints are rarely, if ever, logi-cal (i.e., they are not constitutive of the task).2

C. Size and complexity of problems. Design problems are generally largeand complex spanning time scales on the order of days, months, or evenyears.

I First author.2 These important distinctions have not been widely app reciated in the literature. Simon

(1973a), for example, failed to (refused to) recognize them.


8/35


D.

E.

F.

G.

H.

I.

J.K.

L.

Component parts. Being large and complex, design problems havemany parts. But there s little in the structure of designproblems o dic-

tate he ines of decomposition.Decomposition s substantiallydictatedby the practiceand experience f the designer.Interconnectivity of parts. The components f design problemsare notlogically interconnected. here are, however, many contingent nter-connections mong hem.Right and wrong answers. Designproblemsdo not have ight or wronganswers, nly better and worse ones Rittel & Webber, 1973).Input/output. The input to design problems consists of information

about he people who will use he artifact, thegoals hey want to satisfy,and the behavior believed o lead o goal satisfication.The output con-sistsof the artifact specification.Functional nformation in many waysmediatesbetween he input and output information. This is a ratherstandard characterizationadapted rom Wade (1977), and is furtherdiscussed n Section4.1.3.Feedback loop. There s no genuine eedback rom theworld during theproblem-solvingsession, t must be simulated or generated by the

designer uring the problem-solving ession. eedback rom the worldcomesonly after the design s completed and the artifact is constructedand allowed o function in its intendedenvironment.But, of course,atthis point, the feedback annot nfluence he current project, only thenext “similar” project.Costs of errors. Thereare costs associated ith each and every actionin the world, and the penalty for being wrong can be high (Rittel &Webber, 1973).

Independent functioning of artifact. The artifact is required o functionindependently f the designer.Distinction between specification and delivery. There s a distinction tobe made between he specification of the artifact and the constructionand deliveryof the artifact.Temporal separation between specification and delivery. There s a tem-poral separation etween he specification and deliveryor constructionof the artifact. The specificationprecedes elivery.

Theseareall significant nvariants n the ask environments f prototypi-cal design situations. Many of them have been noted previouslyby otherresearchers. ur claim here s that we can use hem as a template o identifyother cases of design.To the extent that the task environment of a givenproblem situation meets or conforms o this template, hat problem situa-tion is a prototypicalexampleof a design ituation. To the extent hat a taskenvironmentvaries rom this template-by omission of one or more of therequirements-it is a less centralcase of designactivity.


9/35

GENERIC DESIGN 403

Note that we are not stipulating what is and what is not a design activity.To do that we would have to insist that the 12 task environment characteris-

tics listed above, or some subset of them, constitute necessary and sufficientconditions for design activity. We make no such claim. Rather, all we aresuggesting is that we have a template of some salient characteristics com-mon to the task environment of problem situations that are consistently rec-ognized as good examples of design activity. Problem situations in whichthe task environment fails to conform to this template on one or morecounts are deviations from the central case. In this article we will only be in-terested in central cases. We have no interest in saying how far a problem

can deviate from the prototype and still be considered design. Thus, we willuse the label “design” to refer to situations that closely conform to theprototypical or central cases.

Some problem-solving situations, which fit well into the schema, are in-structional design, interior design, some cases of software design, and musiccomposition. Some tasks that deviate slightly are writing and painting. Inthese latter cases there is usually no separation between design and delivery.The problem solver actually constructs the artifact rather than specifying it.

Some activities that deviate more radically are classroom teaching, sponta-neous conversation, and puzzle solving.To illustrate how nondesign tasks differ from design, we will examine the

task environments of two well-structured problems that have been exten-sively studied in the literature (Newell & Simon, 1972). The first task iscryptarithmetic, a puzzle in which an addition problem is presented, but alldigits have been replaced by letters. The task involves solving the additionproblem by making hypotheses about the correspondences between letters

and digits. The second task is the Moore-Anderson logic task, posed as agame in which subjects transform given symbol strings into goal symbolstrings according to certain transformation rules. The game moves areisomorphic to theorem proving by logical inference. The following is a sum-mary comparison of these two tasks to the 12 features of design environ-ment listed previously.

A. ’ Distribution of information. In the case of cryptarithmetic, the startstate is completely specified. The goal test is also clearly defined. Thetransformation function is restricted to only two operations, whichare specified in advance: (a) assign a digit between 0 and 9 to a letter,and (b) perform addition. In the Moore-Anderson tasks both the startand goal states are completely specified. The set of legal operators isalso specified. All the subject has to do is figure out the right order ofapplication.

B. ’ Nature of constraints. In both cryparithmetic and the Moore-Andersontask the rules are definitional or constitutive of the task. As such they


10/35

404

c. ’

D. ’

E. ’

F. ’

G. ’

H. ’

I. ’

J. ’K. ’L. ’

GOEL AND PIROLLI

have a logical necessity about them. If one is violated, we are simplynot playing that game, but some other game.

Size and complexity. In both cryptarithmetic and the Moore-Andersontask, the problems are relatively small and simple. The ones we lookedat (Newell & Simon’s, 1972) took anywhere from 10 to 40 min to solve.Component parts. Even though the problems are relatively small, theybreak down into a number of components. In cryptarithmetic each row(addend) is treated as a component. In the Moore-Anderson task eachwell-formed formula (wff) constitutes a component part. However, inboth cases, unlike the design cases, this breakdown is enforced by the

logical structure of the problem.Interconnectivity of parts. In cryptarithmetic the few components(rows) that do exist, are logically interconnected. That is, there existsthe possibility that any row will sum to greater than 9 and affect thenext row. In the Moore-Anderson task there are logical connectionsbetween the wffs but they are not obvious. In fact, the task is substan-tially to discover these interconnections.Right and wrong aMWerS. In the cryptarithmetic problems we examined

there was only one definite right answer. All other answers were simplywrong. In the Moore-Anderson task the answer consists of the rightsequence of operators. Although there may be several right sequences,all other\s are definitely wrong. Also, it is clear when one has a rightsequence.Input/output. In cryptaritbmetic the input is limited to letters, num-bers, and the rules of the game. The output is a particular sequence ofnumbers. In the Moore-Anderson task the input consists of the wffs,

and the operators. The output consists of the sequence of lawful opera-tor applications sufficient to transform the premise wffs into the con-clusion wff.Feedback loop. In cryptarithmetic there is genuine feedback after everyoperation (i.e., substitution and summation). It is, however, local feed-back and the final decision needs to satisfy global constraints. In theMoore-Anderson task there is local feedback after every operator ap-plication. But it is of limited value. More useful feedback comes after

sequences of operator applications.Costs of errors. In both cases, the cost of error is negligible in the sensethat a wrong answer may cause the subject some embarrassment, but itwill not affect the lives of third-party “users.”Independent functioning of artifact. Not applicable.Distinction between specification and delivery. Not applicable.Temporal separation of specification and delivery. Not applicable.

The reader will note that these nondesign task environments differ from thedesign task environment on all 12 features. Incidentally, they also differ


11/35

GENERIC DESIGN 405

from eachother n respect o features A ’ and F ‘. But the consequences fthis latter differencewill not bepursuedhere. The consequences f the dif-

ferencesbetweendesign task environments and nondesign ask environ-ments will be consideredn some detail in a later section.This is not to suggest hat all nondesign ask environments will be so

radicallydifferent. For instance, n ntermediate ase s providedby the askenvironmentof some mathematical problems, such as those presented nSchoenfeld 1985). However,we have purposefullychosen o contrast hedesign ases with cases hat deviate adically. The sharp contrast makes hedistinctionsclear. f something ubstantive s uncovered,we can move onto

the subtler comparisons hat will be involved n the less deviant cases. fnothing nteresting mergesn these lear-cut ases, othing will emergen thesubtlercases.

3.3 The Structure of Design Problem SpacesThe following is a list of a dozen nvariants ound in the structure of thedesignproblem spaceswe examined.Each can be explained or justified byan appeal o the structures of the nformation-processing ystemand design

task environment as articulated above.1. Problem structuring. The lack of information in start states, goals

states, and transformation functions will require extensive problemstructuringbefore problem solving can commence.

2. Distinctproblem-solvingphases. Designproblemsolvingcan be urthersubcategorized nto three interestingly distinct phases, preliminarydesign, efinement, and detail design.)This is probably due o the sizeand complexity of problemsand the many different types of informa-tion and levelsof detail that need o be considered.

3. Reversing direction of transformation function. Because he structureof the task s not well specified n advance, nd he constraints arenon-logical, the designer as he option to reverse he directionof the trans-formation function by transforming he problem o one or which he orshe may alreadyhavea solution, or into a problem for which the solu-tion is in some way more effectiveor desirable.

4. Modularity/decomposability. Given the size and complexityof designproblems and the limited capacity of short-termmemory, one wouldexpectdecomposition f the problem nto a large number of modules.However, given the fact that there are few or no logical connectionsamong modules but only contingentones, one would expect he de-signer o attend to some of these and gnore the others.

a There is nothing deep about there being three phases rather than n phase s. Designers usethese three phases to talk about their proce sses. W e too found them useful for our purposes. Itwould certainly have been possible to do a finer-grained or coarser-grained individuation.


12/35


5. Incremental development of artifact. Interim design ideas are nurturedand incrementally developed until they are appropriate for the task.

They are rarely discarded and replaced with new ideas. The principlereasons for this would be the size and complexity of problems and thesequential nature of the information-processing system, and the factthat there are no right or wrong answers.

6. Control structure. Designers use a limited commitment mode controlstrategy that enables the generation and evaluation of design compo-nents in multiple contexts.

7. Making and propagating commitments. Because design plans and

specifications have to be produced in a finite amount of time, and haveto be interpretable by a third party, designers have to make, record, andpropagate commitments.

8. Personalized stopping rules and evaluation functions. Because there areno right or wrong answers and direct feedback is lacking, the evaluationfunctions and stopping rules that designers use will be personalized (i.e.,derived from personal experience and immersion in the profession).

9. Predominance of memory retrieval and nondemonstrative inference.

Because there are very few logical constraints on design problems,deductive inference plays only a minimal role in the problem-solvingprocess. Most decisions are a result of memory retrieval and nondeduc-tive inference.

10. Constructing and manipulating models. Because design typically occursin situations where it is not possible, or too expensive to manipulate theworld directly, designers usually manipulate representations of theworld. (We only get one run on the world, whereas we can get as many

runs as we like on models of the world.)11. Abstraction hierarchies. The qualitative difference in the input and out-

put information and the several distinct problem-solving phases resultin orthogonal abstraction hierarchies.

12. Use of artificial symbol systems. Given the size and complexity of prob-lems, the need to construct and manipulate external models, and the useof abstraction hierarchies, designers will make extensive use of artificialsymbol systems.

The first six of these invariants will be discussed and illustrated with data insection 5.0. A lengthier and more complete discussion of all 12 invariants isavailable in Goel(1991), but before beginning the discussion we present ourmethodology and data analyses.

4. DATABASE AND CODING SCHEME

Our empirical studies have focused largely on the analysis of verbal proto-cols from expert designers in various disciplines. This has required the


13/35

GENERIC DESIGN 407

development of a rather complex protocol analysis scheme that can be tiedto our framework for the design problem space. In addition to developing

analyses to find commonalities across design tasks, we have also been in-terested in exploring how design tasks might differ from nondesign tasks.Consequently, we have also performed analyses of nondesign verbal proto-cols that were readily available in the literature. Altogether, we have accu-mulated a database of 12 design protocols and six nondesign protocols.Both sets are described and a detailed analyses of three of the design proto-cols and two of the nondesign protocols is presented.

4.1 Design ProtocolsTwelve design protocols, of approximately 2 hours each were collected fromexperts in the disciplines of architecture, mechanical engineering, and in-structional design. Although we examined and coded all 12, we will restrictour discussion here to one from each discipline. The three were selected onthe basis of (a) the completeness of the artifact specification produced bythe individuals, and (b) the fluency of the subjects’ verbalization.

4.1.1 Subjects. One of the protocols was produced by a subject (SubjectS-A) performing our architecture task. Subject S-A was a doctoral studentin the Department of Architecture at the University of California, Berkeley,who volunteered to participate in the study. Subject S-A had 6 years of pro-fessional experience, but had never designed a post office. A second proto-col was produced by Subject S-M in solving our mechanical engineeringtask. Subject S-M was a doctoral student in the Department of MechanicalEngineering at Stanford University, and also volunteered to participate in

the study. Subject S-M had 3 years of professional experience includingworking for a firm in Italy designing bank teller machines. The third proto-col was collected from Subject S-I on an instructional design task. SubjectS-I was a professional instructional designer working for a large multina-tional corporation who also volunteered to participate in the study. SubjectS-I had over 10 years of experience in designing technical training material,mainly on the operation and servicing of office machinery and systems.Subject S-I was also familiar with the text editor that was the focus of his

design task.

4. I.2 Task Descriptions.

l Architecture. The architecture task involved the design of an auto-mated post office (where postal tellers were replaced by automatedpostal teller machines, APTMs) for a site on the UC Berkeley campus.Subject S-A was given two documents: an outline of the experimentprocedures and a design brief that motivated the need for the post officeand specified the client’s needs and requirements. The site was visiblefrom the place of the experiment.


14/35

TABLE 2

Breakdown of Knowled ge Sources for Statements Made in Each Design Development Phase

Problem Solving

Subiect S-A

Design Brief

Experimenter

Self

Inferred

Subiect S-M

Design Brief

Experimenter

Self

Inferred

Subject S-l

Design Brief

Experimenter

Self

Inferred

Combined Subfects

Design Brief

Experimenter

Self

Problem Prelimlnory

Structuring Design

.09 .oo

.13 .Ol

.77 .96

.Ol 33

.ll .w

.44 .04

.44 .93

.W .03

.30 .W

.14 .05

.47 -95

.Ol .W

.21 .W

.25 .03

.53 .95

Refinement

34

.w

30

.00

.w

.w

1.00

.W

.W

.Ol

.99

.W

.Ol

.W

.96

Detail

Design

.Ol

.w

.93

.06

.w

.oo

1.00

.W

.W

.W

1 .w

.W

.W

.W

.98Inferred .Ol .02 43 -02

TABLE 3

Output Mode Associated with Statements in Each Design Development Phase

Problem Solving

Subfect S-A

Verbal

Written

Subject S-M

Verbal

Written

Subfect S-l

Verbal

Written

Combined Subjects

Verbal

Written

Problem Preliminary

Structuring Design

.06 34

.14 -16

.a9 .55

.ll .45

30 .55

-20 .45

.0B .77

.12 .23

Refinement

.50

.42

.52

.40

.60

.40

.61

.39

Detail

Design

.67

.33

.66

.34

.19

-01

.71

.29

416


15/35

GENERIC DESIGN 409

Focus

Experimentaltask

Monitor Design Miscellaneousdevelopment

Design Design Book

operator method keeping

nProilem

structuringProAlemsolving

/Areliminary Reline Detail

Figure 1. Categories of genera l focus in protocol coding scheme

pointsbeingmadeabout he opic. A second ay of demarcating tatementssto use noncontent ues uch as pauses, hrase and sentence oundaries, ndthe making and breaking of contactbetweenpen and paper. We ooked atboth of thesewaysof demarcating tatements nd appliedwhichever nepro-vided he finer-grainedndividuation. The meanduration of statements asapproximately8 s. The mean number of wordsper statementwasabout 15.

4.1.3.2Coding Scheme. Our protocol codingschemewassimilar n spirit,but not detail, to schemes employed n other recent studies of design(Greeno,Korpi, Jackson,& Michalchik, 1990;Ullman et al., 1988).Eachstatementwas codedalongseveral imensions see Goel, 1991,AppendixD,for a more complete descriptionof the schemealong with illustrative ex-amples).Figure 1 llustrateshow eachstatement ould be assigned o one offour categories sed o identify the general focus of a designer’s ctivity.Experimental task statements were statements concerned with the ex-perimentaldesignand setup. Monitor statements ndicatedmetacogmtion,or reflection about designmethods. Thesewerestatements n which a sub-ject reviewed r commentedupon the problem-solvingm-ocesstself. Threedifferent typesof monitoring statementswere dentified: explicit mentionsof design operators, statements bout design methodology, and statementsabout bookkeeping. Design development statementswere statements hatadvanced he development f the artifact.


16/35


Design development statements were further divided into (a) problem-structuring statements, which generated or solicited information that fur-

ther structured the problem, or (b)problem-solving statements, in which thedesign specification was advanced in some way. Problem-solving statementswere divided into (a) preliminary design statements, which were the initialspecifications of a design solution; (b) refinement statements, which elabo-rated an established design element; and (c) detail statements, which servedto finalize the form of some design element. These labels are rather stan-dard terms that designers routinely use to talk about various phases ofdesign development. They are, of course, relative to the available time and

the degree of completeness of the design specification. But nonetheless,given the final output of the designer, it is possible to trace the developmentand segment it into phases.

We also coded all design development statements according to the aspectsof design development referred to in the statements. This subcategorizationwas orthogonal to the breakdown of design development statementspresented in Figure 1, and differentiated the type of content attended to bythe desiger. For each statement we identified whether the content of the

statement referred to: people, purposes, behavior, function, structure, andresources. As with the problem-solving categories, these subcategories arealso quite standard in the design literature. The ones we employ areadopted from Wade (1977). Briefly, the intuition behind these terms isthat artifacts are designed to perform certain functions, which are calcu-lated to support certain behaviors, which help in the realization of certainpurposes held by people. This categorization provides a chain linking usersto artifacts and recognizes that each intermittent step needs to be con-

sidered. To these categories we have added resource (e.g., time, money,manpower, etc.). It should be noted that these are not disjunctivecategories. A single statement can fall into more than one category.

In addition to coding the general aspects of the design being attended to,we also aggregated statements into episodes of closely related statementsunited by a focus on some particular component of the designed artifact.We called these aggregates modules and submodules. Unlike the codingsdiscussed so far, the module coding scheme was dependent on the particular

design tasks and the particular individual solving the task.Finally, for each statement, we coded some information about the partic-

ular problem-solving step being executed by a designer: (a) the operatorapplied, (b) the content to which it was applied, (c) the mode of the output,and (d) the source of knowledge used (see Figure 2). Operators were a label-ing of statements by the function they served in the problem space.Although we made no theoretical commitment to any specific set, we foundthe 11 noted in Figure 2 adequate for our needs. The mode of output of a

statement was encoded as either verbal or written. Each statement was also


17/35

GENERIC DESIGN 411

Problem solving step

- add- evaluati- propose- comment- repeat- elaborate- lUW- modify

.- VW- read- request- miscellaneous

written verbal sell inleresignbriefF

experimenter

Figure 2. Localized problem-solving step categories in protocol coding scheme

coded or the source of knowledgeof the statement. The four categoriesusedwere he experimenter, the design brief, self (retrieved rom long-termmemory), and inferred (deductively) rom the information existent n the

problem space.Although the completecoding scheme was complex, each part of thescheme rovided a different view onto the unfolding of the designprocessand the evolution of the design tself. In the next subsection, we describehow the scheme was modified to code protocols collected n nondesigntasks.

4.2 Nondesign Protocols

Our six nondesign rotocols weregathered rom several ublished sources.They includemathematics, ryptarithmetic,and the Moore-Andersonasks,and range n duration from 15 min to 40 min. Two mathematicsprotocolswere aken rom Schoenfeld 1985,Appendices .1 and 9.5).These wo wereselected rom several n Schoenfeld because hey were the only ones hatapproximated ur experimental etup of singlesubjects ivinguninterruptedand unpromptedprotocols.Two cryptarithmeticand wo Moore-Andersontask protocols were chosen rom Newell and Simon (1972, Appendices

6.1, 7.1, 9.2, 8c 10.1). These protocols were chosen on the basis of theirduration.Each of these ix nondesign rotocols wasanalyzed nd coded.However,

keepingwith our strategyof contrastingextreme ases, we discuss nd com-pare only the cryptarithmetic and Moore-Anderson ask problem spaceswith the designproblem space.

4.2.1 Subjects. The subjects n both the cryptarithmetic and Moore-

Anderson asks wereundergraduate ollege students.


18/35


4.2.2 Task Descriptions. The cryptarithmetic tasks for both subjects(NS6.1 and NS7.1) involved solving the following puzzle:

DONALD D=5+GERALD

ROBERT

in which each letter stands for a specific digit and the digits associated withROBERTare the sum of the digits associated with DONALD + GERALD,and itis given that D = 5. Cryptarithmetic problems are basically constraint satis-faction problems.

The Moore-Anderson task is a string transformation task, isomorphic todeductive inferences in propositional logic using logical inference schemas.The task for Subject NS9.2 in Newell and Simon (1972) was

Ll: (R - - P) *(-R - Q)LO: -(-Q *P)

where Ll was a premise and LO was a goal. The transformation rules arespecified in Newell and Simon (p. 406). Using the same transformationrules, the task for Subject NS10.2 in Newell and Simon was:

Ll: P * (Q *R)

L2: -(P - T) - -(P * Q)

LO: T .T

where Ll and L2 were premises and LO was the goal.

4.2.3 Protocol Coding Procedure. The protocols were reproduced fromtheir original source and recoded with a subset of the scheme devised for thedesign protocols, which was modified as follows:

1. As with the design protocols, design development statements were dif-ferentiated into problem-structuring and problem-solving statements.However, the problem-solving statements were not further differen-tiated into the various design phases.

2. The aspect of design development category was eliminated.3. The mode of output was not coded.

The first two changes were necessitated by the data. There is no interestingsense in which the nondesign data were amenable to a further breakdown ofthe problem-solving category into subcategories, and the aspect of designdevelopment category was inappropriate. Both of these issues will be dis-cussed later. There was not enough information available in the publishedsources to code for the mode of output.


19/35

GENERIC DESIGN 413

5. THE DESIGN PROBLEM SPACE

In this section, we list and briefly discuss some of the more interesting in-variants in the design problem space. In each case we ask the following threequestions: What is the phenomenon? Why does it occur? What is the sup-porting data? We will also consider if and in what form the phenomenontranspires in nondesign problem spaces.

5.1 Problem StructuringProblem structuring is the process of drawing upon our knowledge to com-

pensate for missing information and using this knowledge to construct theproblem space (Simon, 1973a). It occurs for the obvious reason. Designproblems are incompletely specified but the specification of a problem spacerequires complete information about start states, goal states, operators, andevaluation functions. In Goel and Pirolli (1989) we examined the form andorganization of some of this knowledge and how it was applied. Here wewant to note the extent and location of problem-structuring phases and alsosay a few words about how they differ from problem solving.

The first thing to note is that problem structuring accounted for approxi-mately 25% of the statements devoted to design development in our designprotocols. In our three protocols it ranged from a low of 18% to a high of30% (for a mean of 24%). In contrast, only 0.3% of the nondesign proto-cols were devoted to problem structuring. The second point is that problem-structuring statements occurred mainly at the beginning of the task, whereone would expect them, but also reoccurred periodically as needed. Figure 3shows the temporal distribution, aggregated over 5-min intervals, of the

problem-structuring and problem-solving phases for subjects S-A, S-M,and S-I (see p. 413).Although problem structuring is a widely recognized concept, it is often

unclear as to how it differs from problem solving. In our design protocols,we noted several phenomena that appear to differentiate problem-structur-ing activities from problem-solving activities:

1. Aspects of the design considered. Table 1 (p. 415) presents a breakdownof the aspects of design attended to by subjects across the design devel-opment phases. In general, subjects produced proportionately morestatements about people, the purposes of the artifact, and resourcesduring the problem-structuring phases than in problem-solving phases(preliminary design, refinement, and detail design). In contrast,statements about the structure or function of the artifact were moreprominent in the problem-solving phases than in the problem-structuring phases. These results suggest the problem structuring indesign is associated with attention to how the artifact may be used and


20/35


I wroblem Structuring WPreliminaty Design a Reline men ln Detail Design

5 15 25 35 45 55 65 75 85 95105115

5 15 25 35 45 55 65 75 85 95

Time (minutes)

Figure 3. Distribution and extent of problem structuring and problem solving far Subjec

S-A, S-M, and S-l (aggregated aver 5 min intervals)

what is available to form it, whereas problem solving is associated wilattention to specification of the function and form of the artifact.

2. The primary source of knowledge. The client and design brief were ir

portant sources of knowledge during problem structuring but not duing problem solving (Table 2, p. 416). Problem structuring is associatewith bringing new information into the problem space.


21/35

GENERIC DESIGN 415

TABLE 1

Proportion of Statements Mad e

in Each Design Developm ent Phase Abou t Various Aspects of a Design

Problem

Structuring

Problem Solving

Preliminary Detoil

Design Refinement Design

Subfect S-A

People

Purpose

Resource

BehaviorFunction

Structure

Subiect S-M

People

Purpose

Resource

Behavior

Function

Structure

Subject S-l

People

Purpose

Resource

Behavior

Function

Structure

Combined Subjects

People

Purpose

Resource

Behavior

Function

.14 .13

.14 .02

-11 .02

.OB .ll

.lO .lB

.43 .54

.22 .07 .22

.04 .Ol .oo

.09 .03 .w

.lO .ll .03

.26 .35 .06

.29 .43 .69

.33 -10

.16 .02

.26 .w

34 .02

.oo .15

.20 .71

.22 .ll

.lO .02

.14 .02

.OB .09

.14 .23

.06

.w

.w

.Ol

.lO

33

.w

.w

.oo

.oo

.4B

.52

.04

.W

.W

.Ol

.32

.Ol

.W

43

.Ol.05

.90

.06

.W

.W

.09

.lO

.76

.W

.W

.W

.W

33

.67

.02

.W

.Ol

.03

.19

Structure .32 .53 .63 .75

3. The degree f commitment made o output statements as evidenced ythe quantity and character of written output). There was a higher per-centage of verbal-only statements generated during problem structuringthan during problem solving (Table 3, p. 416). Written output may indi-cate a high degree of commitment to particular design decisions, whereaspurely verbal statements about the design indicate less commitment.

4. Operators. here was a higher percentage of add and propose operatorsin the problem-structuring phase than the problem-solving phase;42.3% of the operators applied during the problem-structuring phasewere add operators. This was systemically reduced to 36.7070, 35.3%,and 32% during the preliminary design, refinement, and detailing


22/35

TABLE 2

Breakdown of Knowled ge Sources for Statements Made in Each Design Development Phase

Problem Solving

Subiect S-A

Design Brief

Experimenter

Self

Inferred

Subiect S-M

Design Brief

Experimenter

Self

Inferred

Subject S-l

Design Brief

Experimenter

Self

Inferred

Combined Subfects

Design Brief

Experimenter

Self

Problem Prelimlnory

Structuring Design

.09 .oo

.13 .Ol

.77 .96

.Ol 33

.ll .w

.44 .04

.44 .93

.W .03

.30 .W

.14 .05

.47 -95

.Ol .W

.21 .W

.25 .03

.53 .95

Refinement

34

.w

30

.00

.w

.w

1.00

.W

.W

.Ol

.99

.W

.Ol

.W

.96

Detail

Design

.Ol

.w

.93

.06

.w

.oo

1.00

.W

.W

.W

1 .w

.W

.W

.W

.98Inferred .Ol .02 43 -02

TABLE 3

Output Mode Associated with Statements in Each Design Development Phase

Problem Solving

Subfect S-A

Verbal

Written

Subject S-M

Verbal

Written

Subfect S-l

Verbal

Written

Combined Subjects

Verbal

Written

Problem Preliminary

Structuring Design

.06 34

.14 -16

.a9 .55

.ll .45

30 .55

-20 .45

.0B .77

.12 .23

Refinement

.50

.42

.52

.40

.60

.40

.61

.39

Detail

Design

.67

.33

.66

.34

.19

-01

.71

.29

416


23/35

GENERIC DESIGN 417

phases espectively.Similarly, proposeoperators,which accounted or10.7% of the operators applied during problem structuring, weresys-

temically decreased o lo%, 7.6%, and 6.7% during the preliminarydesign, efinement, and detailing phases, espectively.

5.2 Distinct Problem-Solving PhasesIn addition to the distinction betweenproblem structuring and problemsolving, there s a further differentiation of problem solving nto severaldistinct phases.We have subcategorized roblem solving nto preliminarydesign, efinement, and detail design. As noted earlier, hese categories re

quite standard among designers. s might be expected, hese phasesweregenerallyengagedn sequentially y our subjects, starting rom preliminarydesign,passing hrough refinements,and endingwith detail design, houghit was not unusual or a subject o return to an earlier phase as previouslyunnoticedaspects merged see igure 3, p. 414).Therewas, however, on-siderablevariability among subjectsas to the amount of time devoted oeach phase.

The threeproblem-solving hases iffer at least n terms of the following

three respects:1.

2.

3.

Aspects of the design considered. There was a steady decrease n theconsideration f people,purpose,and resource spects f design devel-opment rom preliminary to detail designand a corresponding ncreasein the structuralaspect see able 1). The behaviorand unction aspectson average eemed o stay relatively constant across he three phases.The primary source of knowledge. There was still some nput from theclient and/or designbrief at the preliminary designstage, but it disap-peared by the detailing stage see Table 2).The degree of commitment made to output statements (as evidenced bythe quantity and character of written output). There was a steady n-crease n the number of verbalizations ommitted to paper as subjectsprogressed rom preliminary design, through refinement, to detaildesign see able 3). Therewasalso an ncrease n the degree f explicit-ness and detailing n the written or drawn material (see Goel, 1991 orillustrations). We take both of these as evidence f increasing ommit-ment to the emerging design.

It is our conjecture hat the distinct phases esultnot only from the sizeandcomplexity of the problems,but perhaps more importantly, from a com-bination of the different typesof information that must be considered ur-ing the session i.e., people, purposes,behavior, function, and structure)and the different levelsof detail at which it is considered.

Again, the situation was quite different for the nondesign rotocols. n-

stead of the problem solvingbeingcomprisedof distinct phases f different


24/35


activities, it was comprised of cycles of the same basic activity. In nondesignproblems, the subject is searching for a solution. If it is not on the path being

searched, one must back up and start down another path. The activity oneengages in as one goes down each path is basically the same. This is demon-strated by data that will be discussed later.

5.3 Reversing the Direction of Transformation FunctionThe designer naturally interprets the problem situation through personal ex-periences and biases. But in addition to this, designers will occasionally stopand explicitly try to change the problem situation so it more closely fits their

expertise, knowledge, and experience. This involves manipulating both theproblem constraints and the client’s expectations. We call this reversing thedirection of the transformation function because rather than transforminginitial problem states to a goal state, the designer can negotiate changes tothe initial state and goal state that experience suggests are more easilyachievable, or perhaps might lead to a more effective design solution.

An example of an (unsuccessful) attempt to change explicitly the startand goal states specified by a design brief was provided by the following

negotiation sequence. Subject S-A was standing on a ninth floor balconyand had a bird’s eye view of the small triangular site he had been given forthe proposed post office. He was not content just to build a post office butwanted to redesign the whole area.

S-A: So, given the fact we have that triangle over there as a limit. And I can-not exceed that I suppose?

E: Bight, that, that.. .S-A: I have to take that for granted?

E: I, I would think so.S-A: That’s the boundary of. You do not allow me to, to exceed n, in my

area of intervention?E: No, I think you should restrict it to that.

S-A: So, I am constrained to it and there is no way I can take a more radicalattitude. Say, well, look, you are giving me this, but I actually, I, I’dcome back to the client and say well look, I really think that you shouldrestructure actually the whole space, in between the building. I’ddefinitely do that, if that was the case. You come to me as a client, andcome to me with a triangle alone, I will give you an answer back pro-posing the whole space. Because, I, I think the whole space should beconstructed. So, that there is an opportunity to finally to plan and thatspace hrough those, ah, this building, open up Anthropology and, andplan the three buildings together. So, as to really make ah, this ah, amore communal facility. . .

The reasons why such episodes occur are clear: (a) the problem is incom-

pletely specified, and (b) design constraints are nonlogical and thereforemanipulable.


25/35

GENERIC DESIGN 419

Notice such a sequence simply could not (and does not) occur in nonde-sign problem spaces where the problem constraints completely satisfy the

problem, and indeed are constitutive of the problem. If it did occur-if thesubject requested a change in the problem parameters-we would simplysay that he could not do the assigned problem and was changing it to a dif-ferent problem.

5.4 Solution Decomposition into Leaky ModulesA number of researchers have noted the important role played by decom-position in dealing with complexity (Alexander, 1964; Simon, 1962, 1973b,

1977). However, there is considerable disagreement as to the extent andstructure of decomposition of design problems. Some researchers assumestrict treelike decompositions (Alexander, 1964; Brown & Chandrasekaran,1989). But Alexander (1965) argued that “A city is not a tree; it is a semi-lattice.” There has also been some discussion about the extent of intercon-nections or the encapsulation of the modules.

In Goel and Pirolli (1989) it was noted that designers decomposed thedesign solution into “leaky modules” (i.e., sparsely connected modules)

and had two main strategies for dealing with these interconnections: (a) theyeither blocked the leaks by making functional level assumptions about theinterconnected modules; or (b) they deferred further development of thecurrent module in order to attend to an interconnected module. It was alsostressed that partial interconnectivity (as opposed to total connection ortotal disconnection) was a genuine phenomenon and had to be takenseriously.

The decomposition of a design into maximally independent units

ameliorates the difficulties faced by a limited-capacity, information pro-cessor dealing with typically large and complex design problems. Yet, it isalso a fact about the world that artifactual objects and processes are com-posed of entities that are in fact related to one another in complex ways.Somehow, the design process has to decompose a design to reduce atten-tional loads, yet remain attendant to possibly important interconnectionsamong decomposed modules.

In this study we investigate the density and distribution of these inter-

connections. We divided the protocols into modules (see Section 4) and usedthe mentioning of one module inside another module as an indication ofinterconnections between the modules. Here we report the findings forSubject S-A.

Subject S-A decomposed his solution into 34 modules clustered into fourlarger groups. (We will refer to these 34 modules as submodules and use theterm “module” to refer to the four larger groups.) The four groups (ormodules) were “Site,” “Building,” “Services,” and “APTM.” The sub-

modules within the Site module were things like trees, illumination, circula-tion, and so on. The submodules within the Building modules included, mail


26/35


storage,configuration of plan, roof, location of equipment,and the like.The Servicesmodules ncluded tems such as number of machines, mail

pickup, number of people and service imes, and so forth. The APTMmodule ncluded hings suchas machinecomponents, nterface,and stamp-ing procedure.

Given these 34 submodules, here are 1,122 ogically possible ntercon-nections hat can be made. We found that 7.4% of these connectionswereactuallymade. Furthermore, here s, as might be expected, difference nthe density of connectionsamong the four major groups and among thesubmodules nternal to the groups. The former connectionsare consider-

ably denser han the latter (13.4% vs. 4.5%). These esults support claimsof the near decomposibilityof designsolutions.To compare hese esults with the nondesign roblem spaces we did the

sameanalysison several ondesign rotocols. n the case f cryptarithmeticwe found that subjectsdecomposed he task nto modules correspondingocolumns of letters.Because he problems only had six columns, here wereonly six modules ascompared o over 30 or each of the design asks). But,although he actual numberof modulesweresubstantially ewer, he density

of interconnections mong modules was considerably greater. Using thesameprocedure s beforewe found that the density of interconnectionsorSubject NS6.1 was greater han that of the design problem space. n thecryptarithmeticproblem spaceof SubjectNS6.1, 20% of the possible on-nectionsweremade, as opposed o 7.4% for the design problem spaceofSubjectS-A.

The denser nterconnectivity of the cryptarithmetic modules s exactlywhat onewould expectgiven he fact that cryptarithmeticwas designed s a

multiple constraintsatisfactionproblem and all the constraints are ogical(i.e., theymust be attended o). This is perhaps why suchproblemscan haverelatively ew modulesand still be very challenging. he reason designprob-lems can have many modules and still be tractable s because he intercon-nections are contingent rather than logical. The designer has a greaterdegree f flexibility in determiningwhich ones o attend o and which onesto ignore.

5.5 Incremental Development of ArtifactAs interim design deas or solutions are generated, hey are retained,massaged, nd incrementally developed ntil they reach heir final form.Veryrarely are deas or solutions orgotten or discarded. n other words, n-formation about the state of the design,and associated nowledge broughtinto the designproblem space, appears o increase n a monotonic fashionthroughout he designprocess. his is one of the most robust indings n theliteratureon problemsolving n design Kant dcNewell, 1985;Ullman et al.,

1988).The duration of the incremental development rocess s, to a greatextent, a function of the resources vailable.


27/35

GENERIC DESIGN 421

Thereare a number of factors n the design ask environment hat wouldseem o favor a strategy of incrementaldevelopment. irst, the problems

are arge and, given he sequential ature of information processing, annotbe completed n a single processing ycle. Second, because here are fewlogical constraintson designproblemsand no right or wrong answers,hereis little basis or giving up partial solutionsand starting over from scratch.It makes more sense o continue o develop what alreadyexists.Third, in-crementaldevelopments compatiblewith the generation nd evaluation ofdesign components n multiple contexts, which will be discussed n thefollowing sectionon control structure.

Incrementaldevelopment oes not occur n the nondesign rotocols hatwe analyzed.Although it is true that the nondesign roblems werealso toolarge to be completed n a single cognitive step, there was, nonetheless,different characterabout the progression f knowledge tates and problemstates. More specifically,most of the search paths explored n finding solu-tions are wrong. Whena particular search path s abandoned, much of theinformation associatedwith that path is also abandoned s the problemsolverreturns o a previous knowledge tate and begins a new search ath.

The accumulationof knowledge elevant o a solution does not monotoni-cally increaseas the problem solver switches from one search path toanother.

Incremental development n design, and its comparison o nondesignproblem solving can be better understood by examiningsome particularprotocol analyses nd a more detailed discussionof the control processesoperative n designproblem solving. Both are discussedn the next section.

5.6 Control StructureThere are a number of issues hat a control strategy or traversing designproblem spaces eeds o address. Among them are the following three:

1. Are the solution modules o be developed n isolation from eachother,or are there o be interconnections mong he solutions?

2. Is the information to be processed equentially or in parallel?3. Are the solutions o be developed ncrementallyor appear completely

formed?We have alreadydiscussed esultsand assumptions egarding eachof theseissues: a) the solution modules are nterconnected o some degree; b) thecognitive process s assumed o be sequential; and (c) the solutions aredeveloped ncrementally, A control strategy hat accommodates nd sup-ports each of these acts is required.

Our data ndicate hat designers se a limited-commitment-mode ontrol

strategy (LCM control strategy).This strategy s closely elated o Stefik’s(1980) least-commitment” control strategy.The basic eature of the LCM


28/35


strategy s that, when working on a particular module, t does not requirethe designer o complete hat module before beginning another. Instead,

one has the option of putting any module on hold and attending o otherrelated or even unrelated)modules,and returning o the module on hold ata ater time, This embedding an goseveral evelsdeepand one s not irrevo-cably committed o interim solutions.One alwayshas he option of modify-ing partial designsolutionsat a later point. This, in effect, lets he designertake advantage f multiple problem-solving ontexts n the generation ndevaluationof designelements. o specify his more precisely,we need o (a)show that design elementsare ndeed consideredn different contexts,and

(b) traceout the actual control structureshowing he LCM control strategy.In order to do (a) we first need a way to individuate design elementsanddesigncontexts.

5.6.1 Individuating Design Elements and Contexts. There s a straight-forward mappingamongdesignelements nd the modules and submodulesidentified earlier.Contextsarenot easy o individuate. They were dentifiedusing the following method: The protocol was divided into modules and

submodules, nd a unique numberwasassigned o each oken occurrence fmodulesand submodules. ach numbered egment t the evel of moduleorsubmodule onstituteda different context.Whenevermoduleor submoduletype were nstantiated, t was n a different context.

This notion of context seems o combine both temporal and contentcomponents, ecause henever uniquely numberedmoduleor submodulewas considered, t was at a unique ime t and t was proceeded nd followedby a uniquelynumbered equence f modules or submodules which means

therewasdifferent nformation in the problemspace).Wehave, or each ofthe subjects, a tracing of modules and submodules and the contexts nwhich they wereconsidered. hese racings are best presented n conjunc-tion with control strategies, o which we now turn our attention.

5.6.2. Control Strategies. Control strategies an be enumerated t differ-ent evels. We used a three-level ierarchicalanalyses hat accounts or theprotocols at the level of module, submodule, and individual statement

types. As the module and submodule categories are task- and subject-specific, he control structure at these evels will also be task- and subject-specific. Because he categories t the level of individual statements aregeneralacrossall the tasks and subjects, he control structure at this levelwill also generalize cross asks and subjects.

The actual formalism that we use to capture and display the controlstructure s a transition network; more specifically, t is a recursive ransi-tion network (RTN; Winograd, 1983).These networks have been widely

used n the computational inguistic field to recognize, arse, and generatenatural anguage trings. Weuse hem (manually) o recognize ur protocols.


29/35

GENERIC DESIGN 423

Some samples of control structure from Subject S-A’s protocol (onefrom each level) are presented in Networks 1, 2, and 3 (Figure 4, p. 424).

The reader is referred to Goel (1991) for a representation of the completestructure. The salient features of the networks are summarized here, and thereader is invited to examine Figure 4 for details.

The control structure is naturally analyzed into three hierarchical levels:

Modules (Network 1)

1(reamive call)

1Submodules (Network 2)

1(recursive call)

1Statements (Network 3)

The first two levels are task-specific; the third is general across all threetasks.Within any of the levels, one does not get a consistent, steady, linearmovement through the problem. Rather, one gets repetitive, cyclical,flexible control structure at all three levels.The effect of this repetition and reiteration is that most modules andsubmodules are considered several times, in several different contexts.For example, in Network 1, the site module is considered five times, thebuilding module seven times, and services module four times, and theAPTM module four times. Table 4 summarizes the multiple occurrencesof submodules (see p. 425).The single Local Control Structure (LCS) network can recognize all themodules in all the protocols. That is, it can do a flat or local statementby statement recognition of all the protocols.Most of the action in the protocols seems to be internal to majormodules, at the level of submodules (Network 2). The top- and bottom-level networks (Networks 1, 3) are rather sparse and simple.

Although the control strategy for the nondesign problem space lookssimilar (in terms of the cyclical, repetitive revisitation of modules) to theLCM control strategy of the design problem space (see Goel, 1991) much ofthe similarity is superficial. There is an important difference in the twocases. In the nondesign problem space most of the problem solving occursinternal to modules and/or episodes. There is little carryover from previous


30/35

-se~“p0Luq”s

aq+ 110 ,0 a,“,,“,,S ,O,,“O> [OJOI M.j+ S0Z U6OJaJ E yJOM+JN ‘~~IlpOUJqtlS ,, egs,, aq+ saz u603aJ 2 yJom+aN *salnpoiu

~noj aq+ jo lanai aq, +oIOJO(OJd a4alduroD aq4 sazy602aJ 1 yJom4aN -pasJahoJ( amS~JD ay tp qM u (4xawo3 PUD)

amanbas aq4 @ads sJaqwnu J,, aql ‘Io>o4oJd s,v-s pa qnS az@oDaJ 4~114 syJoM+au Nla jo aldtuos ‘v0Jn6jd


31/35

GENERIC DESIGN 425

TABLE 4

Occurrences of Subm odules in Mu ltiple Contexts for Subject S-A

Modules

No. of SubmodulesSubmodules Considered

Considered Mare Than Once (%I

Range

Hiah Low

Site 9 89 7 1

Bui ld ing 18 20 5 1

Services 4 25 3 1

APTM 3 33 2 1

Total 33 44

TABLE 5

Trace of the Developm ent of Kno wledg e in Subject NS6.l’s Repea ted Visits to Mod ule 6

Visit No.

1.

2.

3.

4.

5.

6.

Concluding Knowledge State Development of Knowledge

G has to be an even number Initial proposal

No letter in front of R Con nection with first visit unclear

G has to be either 1 or 2 Ignore/reverse previous conclusion

R has to be greater than 5 Con nection to previous visits unclear

G is going to be 1 May be connected to Visit 3

R =9 May be connected to Visit 4

G=3 Uncon nected to any previous state

visits to a module or episode. n fact, Newell and Simon (1972), n theiroriginal analysisof theseprotocols, claimed that in returning to a formerstate, he subject s in fact returning o the previous knowledge tate withrespect o the problem. f the subject goesdown he wrong path and returnsto the previousstate, all that the subjectknows s that the path ust exploreddoes not lead to the goal state. The subject does not have an enrichedunderstanding f the state he is returning o.

This can be demonstrated by tracing through the repeat visits to amoduleand/or episode and examining he state of knowledgeat the end ofeach visit. Table 5 provides such a trace of Subject NS6.l’s visits to thesixth column module. Note the third, “development of knowledge,” col-umn in Table 5; it indicates hat there s not a close connection etween hecurrent visit to a module or episode and previous suchvisits.

In the design case, although problem solving does occur internal tomodules, here s also considerable arryover nd development f the modulefrom visit to visit, as evidenced y the incremental development henom-enon, and further substantiated y the trace of SubjectS-A’s repeated isitsto the submodule seating” in Table 6. When he designer ycles back, t is


32/35


TABLE 6

Trace of the Development of Knowled ge

in Subiect S-A’s Repeated Visits to the “Seating” Module

Vlslt No. Concluding Knowledg e State Development of Knowled ge

1. Kee p seating below evergreens: Initial proposol

don’ t disturb overall scheme2. Keep existing seating Reaffirm original proposal

3. Incorporate seating elemen ts OS Builds upon first two visits

part of buil din g strucutre; relaxation

and view of playing field important4. As per third visit, but decouple Modify third visit

seating from buil din g structure

5. Locate seating alon g boarders Bu ild upon fourth visit6. Counterposltion seats so OS to break Build upon fifth visit

up symmetry and not affect circulation

not to the previous knowledge tate, but rather to a previous opic instan-tiated n the current context.This is indicativeof somehigher evel controlstructure hat we have not yet uncovered.

6. SUMMARY AND CONCLUSION

In this article we have proposed framework and method or the nvestiga-tion of designas a cognitive process, and demonstrated ome nterestingresults hey yield. In particular, we started with some ntuitions about thenotion of generic esignand put forward thedesignproblemspace ypothe-sis. To investigate he hypothesis we:

1. Circumscribed esignactivity from nondesign ctivity in a nonarbitraryand nonvacuous manner bya. suggesting hat design s a radical category exhibiting prototype

effects;b. examiningsome prototypecases nd analyzing heir task environ-

ments;C. noting that not all task environments hare hese eatures i.e., not

every ask environment s a design ask environment); andd. associating esign activity with certain nvariant characteristics ftask environments.

2. Used the s tructure of the design ask environment and a few well-accepted onstraints n the structure of information-processingystemsto generate 12 nvariant characteristics f design problem spaces.

3. Analyzed data from both design and nondesign asks and foundevidence or the postulated nvariants. More particularly, we presented


33/35

GENERIC DESIGN 427

data illustrating extensive problems structuring, distinct problem-solving phases, reversing the direction of the transformation function,

near decomposibility of design solutions, incremental development ofdesign solutions, and a LCM control strategy. We were also able to ex-plain why each of the invariants occur by appealing to the structure ofthe design task environment. Along the way we provided a detailedqualitative and quantitative characterization of design problem spaces.

4. Equally importantly, we also presented evidence indicating that theseinvariants do not occur either at all, or at least not in the same form, insome nondesign problem spaces.

On the basis of this data and analyses, we conclude that the notion of adesign problem space is an interesting and explanatory theoretical constructworthy of further study.

REFERENCES

Akin, 0. (1979). An exploration of the design proces s. Design Methods and Theories, 13,

115-119.Akin, 0. (1986). Psycholog y of architectural design. London: Pion.Alberti, L.B. (1988). On theart of building in ten books (J. Ry kw ert, N. Leach, &R. Tavernor,

Trans .). Cambridge, M IT Press. (Original work published 1450)Alexander, C. (1964). Notes on the synthe sis of form . Cam brige, MA : Ha rvard University

Press.Alexander, C. (1965). A city is not a tree. Architectural Forum , 122, 58-62.Alexander, C., & Poyner, B. (1966). The atom s of environmentalstructure. London: Ministry

of Public Building and W ork s.Anderson, J.R . (1983). The architecture of cognition. Cam bridge, MA : Harvard University

Press.Archer, L.B. (1969). The structure of the design proces s. In G. Broadbent &A. Ward (Eds.),

Design m ethods of architecture. New York: Wittenborn.Brown, D. C., & Chandrasekaran, B. (1989). Designproblemsolving. San Ma teo, CA : Morgan

Kaufmann.Chandrasekaran, B. (1983). Towa rds a taxonom y of problem-solving types . AI Magazine, 4,

9-17.Chandrasekaran, B. (1987). Tow ards a functional architecture for intelligence based on generic

information-processing task s. Proceedings of the International Joint Conference on

Artificial Intelligence (pp. 1183-l 192). San Mateo , CA: Morgan Kau fmann.Cro ss, N. (1984). Developm ents in design methodology. Ne w York: W iley.Curtis, B., Krasner, H. , & Iscoe, N. (1988). A field study of the software design process for

large sys tem s.. Com mun ications of the AC M, 31, 1268-1287.Darke , J. (1979). The primary generator and the design proces s. Design Studies, I, 36-44.Dertouzo s, M.L ., Lester, R.K ., & Solow. R .M . (1989). Made in Am erica: Regaining thepro-

ductive edge. Cambridge, MA : MIT Press.Dixon, J.R .. & Du ffey, M .R . (1990). The neglect of engineering design. California Manage-

ment Review, 32, 9-23.


34/35


Eastm an, C. M . (1969). On the analysis of intuitive design proces ses. In G. Moore (Ed.),Emerging techniques in environment design andplanning. Cambridge, MA: MIT Press.

Ericsson, K.A., L Simon, H.A . (1984).Protocol analysis: Verba l reports OS ata. Cambridge,MA: MIT Press.

Gentner, D., & Gentner, D. R. (1983). Flowing waters or teeming crow ds: Mental mode ls ofelectricity. In D. Gentner& A.L. Stevens (Eds.),Mental models. Hillsdale, NJ:Erlbaum.

Gael, V. (1991).Sketches of thought: A study of the role of sketching in design prob lem solv-ing and its implic ations for the comp utational theory of min d.Unpublished doctoraldissertation, University of California, Berkeley.

Goel, V., & Pirolli, P. (1989). Motivating the notion of generic design within information-processing theory: The design problem space .AI Maga zine, IO,19-36.

Greeno, J .G. (1978). Natures of problem-solving abilities. In W .K. Estes (Ed.),Handbook oflearnin g and cognitiveprocesses. Vol . 5: Hum an informa tion processing.Hillsdale, NJ:Erlbaum.

Greeno, J.G ., Korpi, M ., Jack son, D. , Br’Michalchik, V. (1990). Ill-structured problem solvingin instructional design.Proceedings of the Annua l Codefence of the Cognitive ScienceSociety (pp. 939-946). Hillsdale, NJ : Erlbaum.

Guindon, R. (1989). The process of knowledge discovery in sys tem design.Proceedings of theInternationa l Cortference on Human-C omputer Interaction.Elsevier.

Jeffries, R ., Turner, A.D ., Polson, P.G., & Atwood, M .E. (1981). The processe s involved indesigning software . In J.R . Anderson (Ed.),Cognitive skills and their acquisition.

Hillsdale, NJ : Erlbaum.Kant, E. (1985). Understanding and automating algorithm design.Proceedings of the Ninth

Internationa l Joint Conference on Artificial Intelligenc e.Los Altos, CA: MorganKaufmann.

Kant, E ., &New ell, A. (1985). Problem-solving techniques for the design of algorithms.Injor-motion Processing and Managem ent, 20,97-118.

Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal about the min d.Chicago: University of Chicago Press.

Liskov , B., Kc Guttag, J. (1986).Abstraction and specification in program developm ent.Cambridge, MA: MIT Press.

March, L.J. (1976). The logic of design and the question of value. In L.J. March (Ed.),Thearchitecture of form. Cambridge, England: Cambridge University Press,

Me llars, P. (1989). Technological changes acro ss the middle-upper Paleolithic transition:Econo mic, social, and cognitive perspec tives. In P. Mellars & C . Stringer (Eds.),Thehum an revolution: Behavio ral and biolo gical perspectives on the origins of modern

hominid% Edinburgh: Edinburgh University Press .Mo stow , J. (1985). Toward better mode ls o f the design proces s.AI Magazine, 6, 44-57.Newell, A. (1990).Unifie d theories of cognition. Cambridge, MA: Harvard University Press.New ell, A., & Simon, H.A. (1972).Human problem solving. Englewood Cliffs, NJ : Prentice-

Hall.Perkins, D.N. (1986).Knowled ge OS design. Hillsdale, NJ : Erlbaum.Reitma n, W .R . (1964). Heuristic decision procedures, open constraints, and the structure of

ill-defined problems. In M .W . Shelly& G.L. Bryan (Eds.),Human judgements andoptimolity. Ne w York: Wiley.’

Rittel, H ., & Web ber, M . (1973). Dilemm as in a general theory of planning.Policy Sciences, 4,155-169.

Ros ch, E. (1978). Principles of categorization. In E. Rosch & B.B. Lloyd (Eds.),Cognitionand categorization. Hillsdale, NJ : Erlbaum.

Schoenfeld, A. (1985).Mathematical problem solving. Orlando, FL: Acade mic.


35/35

GENERIC DESIGN 429

Simon, H .A. (1962). Th

Date post:	07-Jul-2018
Category:	Documents
Upload:	luis-valero
View:	213 times
Download:	0 times

GOEL, Vinod and PIROLLI, Peter. the Structure of Design Problem Spaces

Documents