Abstract Abstraction is a fundamental tool of humanthought in every context. This essay briefly reviews somemanifestations of abstraction in everyday life, in engineer-ing and mathematics, and in software and system develop-ment. Vertical and horizontal abstraction are distinguishedand characterised. The use of vertical abstraction in top-down and bottom-up program development is discussed, andalso, the use of horizontal abstraction in one very differentapproach to program design. The ubiquitous use of analog-ical models in software is explained in terms of analyticalabstractions. Some aspects of the practical use of abstractionin the development of computer-based systems are explored.The necessity of multiple abstractions is argued from theessential nature of abstraction, which by definition focuseson some concerns at the expense of discarding others. Finally,some general recommendations are offered for a consciouslythoughtful use of abstraction in software development.
Abstraction is a fundamental human faculty for perception,action and thought at every level. It is manifested every-where in the growth and exercise of our practical intellectfrom birth. It is a vital tool in the evolution and practice of
Communicated by Prof. Jon Whittle and Gregor Engels.
M. Jackson (B)The Open University, Milton Keynes, UKe-mail: [email protected]
science, mathematics and engineering. The development ofsoftware for programmable digital computers, in particular,presents imperative demands and irresistible opportunitiesfor the exercise and study of abstraction. Writers on soft-ware development [7,16,21] have regarded appropriate useof abstraction as a fundamental skill. This is why a discussionof abstraction is relevant to this special issue of the SoSyMjournal on software development and modelling.
Because abstraction is found everywhere in our intellec-tual landscape, a short discussion can scarcely aim at a com-prehensive survey of its value and practice. In this essay, theapproach is informal, and focuses chiefly on abstraction insoftware development. The discussion falls into three mainsections. Section 2 offers a preliminary account of the devel-opment of certain aspects and uses of abstraction, and men-tions some illustrative examples. Section 3 draws on thisbackground in discussing different dimensions and forms ofabstraction, purposes that it can serve, and the practical intel-lectual structures it can suggest and support. Section 4 exam-ines some critical aspects of abstraction as it is commonlypractised in the development of computer-based systems.A final section draws together some general observations,and recommends a carefully thoughtful use of this funda-mental intellectual tool.
2 A preliminary account
The essence of abstraction is simple and unsurprising: toabstract is to set aside what is less relevant, focusing attentionon what we judge as more important for the purpose in hand.To recognise a persistent entity, we focus on what persists,and abstract away what varies from one encounter to another.We recognise classes of entities by abstracting away the dif-ferences among their members, and by a further exercise of
123
496 M. Jackson
abstraction we may recognise a superclass of some classesalready recognised. We identify a quality common to manyinstances of different kinds, and—in the spirit of Plato—regard the abstract quality itself as an individual thing. Theabstractions that we acquire in our everyday lives, by exer-cising our own faculties and by learning from others in oursociety, furnish—quite literally—our view of the world. It iswell established that the larger part of the scene that we thinkwe see with our eyes is, in fact, supplied by mental activityin which we interpret the purely optical signals in the light ofabstractions and expectations formed from past experience.As Richard Gregory [13] writes: “Without the computingpower and memory that brains bring to bear, retinal imageswould be meaningless patterns of limited use—hence theimportance of knowledge for seeing.”
Abstractions are closely interwoven with theories abouthow the world is and how it works. We equip ourselves witha usable—though not always well-founded—repertoire ofabstractions and associated theories. The theories are basedon what we believe to be the properties and behaviour ofthe material reality from which we have drawn our abstrac-tions. These are grounded abstractions: they spring from ourexperience and perception of the world and our efforts tounderstand it. Ostensibly, the criterion of their validity is anobjective correspondence to material reality; but in truth, thecriterion is how well they work in practice for the purposeswe want them to serve. In early societies, these purposes areprimarily social, rather than intellectual, scientific, or techni-cal, and the criterion of validity is therefore a general socialacceptance. From this intuitive, tacit and informal purpose,repertoires of grounded abstractions grow by unguided evo-lution, and may evolve to embrace such notions as magicspells and witchcraft, or prophecy by examining the entrailsof sacrificed animals. Social acceptance of the associatedtheories—for example, that crop failure is always traceableto the action of a nearby malicious witch—is unquestioned:members of the society “reason excellently in the idiom oftheir beliefs but they cannot reason outside, or against, theirbeliefs because they have no other idiom in which to expresstheir thoughts” [10].
A society that adopts ambitious engineering purposes—building royal tombs, temples, bridges, ships and irrigationschemes—must submit at least some of its grounded abstrac-tions to more stringent tests of objective practical validity.Trade, especially between merchants from different societies,submits the accepted abstractions of value and exchange totests of willingness between traders and of success and fail-ure in each merchant’s ambition for wealth. Large engineer-ing projects demand not only careful attention to predictablebehaviour of static structures but also accurate calculationof the resources necessary for construction and of the taxa-tion that can collect these resources. Engineers, traders andtax collectors are highly motivated to recognise, discover
or to invent new and better grounded abstractions to meetthese more objective criteria of validity. In this way, com-merce, building and land surveying stimulated the beginningsof geometry and arithmetic in ancient Egypt. Babyloniansdeveloped a workable number system, and knew many cal-culational procedures and heuristics. They knew that (3, 4, 5),(5, 12, 13), (65, 72, 97) and many other integer triples corre-spond to the lengths of the sides of right triangles.
These were large practical achievements, applying a sub-stantial repertoire of grounded mathematical abstractions.For the ancient Greeks, the theories associated with math-ematical abstractions had an intellectual interest far beyondtheir immediate practical uses, and they began the consciousprocess by which grounded abstractions could free them-selves from their worldly origins and become objects ofintrinsic mathematical interest and focused study. The crite-ria of success in this study are no longer practical utility andfidelity to a specific physical reality, but elegance, internalconsistency, intellectual richness and a fruitful connection toother mathematical ideas. The evolution of such free abstrac-tions, dissociated from their material ground in the world,naturally tends to increase formality, because formality sup-ports a precision of thought that promises greater certainty inreasoning, and therefore, greater power in developing theo-ries. A clear distinction emerges between grounded and freeabstractions. A grounded abstraction is satisfactory only ifreasoning about the abstraction can be expected to produceresults that are true of the subject in which it is grounded.A free abstraction is not to be judged by this criterion: aproposed non-Euclidean geometry cannot be proved unsat-isfactory by showing that there is no physical reality that itdescribes. For a pure free abstraction, any correspondence tophysical reality is irrelevant. As David Hilbert [29] is reportedas saying:
“It must be possible to replace in all geometric state-ments the words point, line, plane, by table, chair,mug.”
Yet, free abstractions may still be brought to bear on prac-tical non-formal problems. If the objects and relationships ina practical worldly context can be convincingly mapped toobjects and relationships of a free abstraction, then the the-ory of the abstraction may apply—at least approximately—tothe world. The relationship between the free abstraction andthe physical world is described by the mathematician G. H.Hardy [14]:
“The geometer offers to the physicist a whole set ofmaps from which to choose. One map, perhaps, willfit the facts better than others, and then the geometrywhich provides that particular map will be the geome-try most important for applied mathematics. I may add
123
Aspects of abstraction in software development 497
that even a pure mathematician may find his appre-ciation of this geometry quickened, since there is nomathematician so pure that he feels no interest at allin the physical world; but, in so far as he succumbsto this temptation, he will be abandoning his purelymathematical position.”
The effective practical application of mathematics tomaterial reality often demands large efforts of calculation.Calculation was eased by mathematicians’ development ofbetter algorithms, and eventually, by printed tables of log-arithms and other functions. From the seventeenth centuryand even earlier, inventors devised calculating machines toperform simple arithmetical operations: Pascal’s was per-haps the most famous. In the 1820s, Babbage conceivedhis Difference Engine as a machine to calculate and printmathematical tables, and later, he conceived the Analyti-cal Engine as a machine to execute general calculationsspecified by programs; but full working versions of thesemachines were never built. From around 1850, various desk-top machines were devised that could conveniently executeindividual arithmetic operations, but it was not until thedevelopment of digital electronic computers in the 1940sthat general programmed calculations could be effectivelyand reliably mechanised.
Against this background, electronic computers were nat-urally seen at first as a tool for numerical calculations. Theessence of the program’s task was to take the place ofthe human calculator, instructing the computer to performthe desired calculation by specifying an appropriate sequenceof arithmetic operations. Programming was a mathemati-cal endeavour, purely because the work to be mechanisedwas mathematical. Nonetheless, a deeper and more fruitfulview gradually became widely understood and adopted. AsDijkstra [8] wrote, looking back much later: “It used to bethe program’s purpose to instruct our computers; it becamethe computer’s purpose to execute our programs.” Gradually,the digital computer was recognised as a machine that cantake any formal abstraction, suitably expressed in a program-ming language, and exhibit a behaviour in which that abstrac-tion becomes a physical reality, realised in the dynamic fabricof the computer itself.
In the 1950s and 1960s, computers increased in size, reli-ability and speed. Programs became more ambitious, morecomplex, and—not infrequently—impenetrably obscure.Work on program-development method focused on programintelligibility, on specification and correctness, and on pro-gram design. Structured programming advocated an abstrac-tion of control structure that could bring program text andprogram execution into a clearer relationship with each otherand with the problem to be solved. The programming lan-guage Simula 67 [4], motivated as its name suggests by pro-gramming for simulation problems, introduced the idea of
a program component designed to capture an abstraction ofan entity in the world to be simulated, such as a bus or atruck, or a customer in a busy post office: the software com-ponent’s behaviour would reflect the behaviour of the real-world entity.
Simula 67 also encouraged a general view of programdesign as an exercise in designing a structure of abstrac-tions, a view adopted by Dijkstra in the design [6] of theTHE operating system. In this view, there is a conceptual gapbetween the high-level abstractions germane to the problemand the low-level abstractions offered by the programminglanguage. This vertical gap must be bridged by a hierarchyof abstractions, each grounded in the abstractions at the nextlower level. Whether the bridge is built in the top-down orbottom-up direction, or by a combination of both, the pro-posed technique is largely an exercise in inventing abstrac-tions. A design process in the top-down direction, in whichan abstraction is formulated before the ground from which itabstracts has been explored, is a form of what is called refine-ment. If there are multiple levels of design, this is stepwiserefinement.
Stepwise refinement became enormously influential invarious forms, including many manifestations of top-downprogram and system design. Simula 67’s notion of pro-gram components as abstractions of real-world entities waseven more influential, stimulating the development of object-oriented programming languages and of abstract data typesas a vehicle for software specification. Object classes andabstract data types can be designed to capture—though oftenonly approximately—some properties of entities specific toa simulated or modelled reality, such as a bank account or thecomponents of a radiation therapy machine. They can alsocapture free abstractions drawn from the discrete mathemat-ics of programming—such as sets and stacks and queues—that can prove useful in program design.
The same ideas have continued to influence the devel-opment of computer-based systems. These are systems inwhich the computer interacts directly with the material prob-lem world about which it computes and whose behaviour itmust monitor and control. In such a system, the success of thesoftware is judged by whether it evokes the desired behaviourin the world: its development is, therefore, a task cruciallydifferent from programming.
In programming, narrowly understood, the computer isinsulated from the physical reality—if any—of its subjectmatter. The programmer’s task is to satisfy a formal spec-ification of the computation result, expressed as a relationbetween computer inputs and outputs or between a precon-dition and postcondition on program variables. Of course,every computation has some subject matter, which may beabstract or concrete, and the program design must rely onsome abstraction of its subject matter. An abstraction whichembodies a contradiction is likely to cause the program to
123
498 M. Jackson
fail. But a program based on an abstraction which self-consistently misrepresents its subject matter will not fail forthat reason alone: the specification may be ill-chosen butthe program may be correct. Dijkstra [9] saw a formal func-tional specification as a firewall to separate these two distinctconcerns: the ‘pleasantness’ question whether a program sat-isfying that specification is desirable, and the ‘correctness’question of how to design such a program. The program-mer, qua programmer, is not required to consider the variousaspects of the pleasantness question: is the abstraction faith-ful to the reality? Does the computer input correctly representthe state of that reality? What use will be made of the com-puted results?
For a computer-based system, these questions can be nei-ther delegated nor evaded: their answers are tightly integratedwith the development of the software. Because the com-puter is directly interfaced to the problem world by sensorsand actuators, its behaviour at the input-output interface isipso facto a behaviour of some part of the physical problemworld and also a cause of behaviour of other parts. For thewhole functionality of a computer-based system, Dijkstra’sproposed separation is impractical: one might as well try toseparate the two sides of a densely meaningful human con-versation. In such a system, the complexity of the problemworld and of the behaviour that the software must evoke in itpresent a major challenge at every level to the effective useof abstraction.
3 Purposes, dimensions and forms of abstraction
Abstraction, taking many forms and serving many purposes,can be considered from many points of view and structured inmany ways. In this section, different particular perspectivesare adopted to address different particular aspects or uses ofabstraction.
3.1 Vertical abstraction for recognition and theory-building
It is common to speak of levels or layers of abstraction, sug-gesting a vertical dimension. A higher level of abstraction ischaracterised by the possession of concepts of larger granu-larity and greater power—intellectual constructs built uponthe less powerful concepts of a lower level.
A vertical abstraction recognises that a subset of phe-nomena in the subject matter forms a cluster that is highlysignificant for some purpose. The recognition of any coherententity—even the newborn baby’s recognition of its mother—is an exercise of vertical abstraction. Where there are mul-tiple instances of the recognised cluster, the clusters them-selves become phenomena of a new class in our conceptualalphabet, distinct from the constituent lower-level phenom-ena whose associations and relationships they embody. In
Euclidean plane geometry, the notion of a circle is a verti-cal abstraction: it is a plane figure bounded by a closed linesuch that there is one point of the figure—called the circle’scentre—that is equidistant from every point on the line. Thereward of a vertical abstraction is the richness of the associ-ated theory. Many interesting theorems can be proved aboutplane figures constructed of circles and straight lines, andmany useful terms can be defined, such as diameter, radius,tangent, and semicircle. The use of the circle concept, and ofthe associated terminology and theorems, increases the econ-omy, and hence the potential power, of discourse in Euclideangeometry.
Vertical abstraction does not in itself imply either encap-sulation or information hiding [24]: no part of the subjectmatter of the lower level becomes hidden in the higher level.In the example of the circle, the basic constituent phenom-ena of the circle abstraction are the centre point and the set ofequidistant points at the lower level: these are not discardedin the abstraction, but remain visible and directly available,along with all other phenomena of the lower level, as partici-pants in constructions and theorems applying to circles. Thecircle abstraction raises the level simply by introducing anadditional fruitful concept that was not explicitly named andavailable at the lower level. Information hiding and encap-sulation, by contrast, are disciplines for software develop-ment that allow programming abstractions to be devised andused independently of their implementation. They concealthe implementation—which could be considered the groundof the abstraction—by hiding it behind an impenetrable wall.Their purpose is to ensure that users of an abstraction are pre-vented both from damaging the detailed implementation andfrom illicitly exploiting any peculiar contingent properties itmay have.
3.2 Horizontal abstraction for description
The commonest and simplest exercise of abstraction is thepurposeful selection that is inherent in making any descrip-tion. No new concepts or phenomena are introduced, and nonew relationships among those already existing: the descrip-tion is merely restricted to those selected as significant forthe purpose in hand. In contrast to vertical abstraction, wemay call this horizontal abstraction. In terms of abstractionlevels, a horizontal abstraction and its subject matter are onthe same level.
Harry Beck’s famous 1933 map of the London Under-ground system, appositely cited by Jeff Kramer [21], was theproduct of a conscious horizontal abstraction. From 1889,earlier Underground maps [26] had shown the growing net-work of lines and stations superimposed on the backgroundof a conventional street map of London. Over the follow-ing 30 years this background of streets became graduallyfainter and less detailed in successive maps, and in 1920 it was
123
Aspects of abstraction in software development 499
abandoned altogether. But for the next 10 or 12 years, evenmaps in which no streets were shown still placed the lines andstations with precise topographical accuracy, against a faintlydepicted background of a few famous landmarks. Beck, per-haps influenced by his work as an electrical draughtsman,decided that this topographical accuracy was unimportant.His Underground lines ran straight up and down the map, oracross it, or on 45◦ diagonals, and stations and lines werespaced for maximum clarity.
Beck had rightly distinguished two subsets of phenom-ena relevant to the Underground users. The first subset con-tains the phenomena of precise geographical location, whichallowed users to identify the stations nearest to their destina-tion and starting points and to estimate the journey distance asthe crow flies. The second subset contains the phenomena ofstations sequenced along each line and of line intersectionsat certain stations, which allowed users who already knewtheir destination and starting stations to plan a journey thatminimised the number of intervening stations or the numberof changes from line to line. In his abstraction, Beck judgedthe second purpose to be more important than the first: so hediscarded the geographical phenomena to allow the journeyphenomena to be shown in the clearest and most useful way.Planning a journey with his map usually proved very easy.The map was a brilliant success, and its design was copied forrail transport systems in many other countries. Harry Beckpersonally continued until 1960 to produce maps of the con-tinually evolving network, and today’s Underground mapsstill adhere to the basic principles of his design.
Horizontal abstraction of this kind is often applied to aninstance—in this case, the London Underground system—rather than a class. Its product is a description of the subjectmatter. We might say that it is the purest form of abstraction,because its essential effect is to discard some part of the sub-ject matter: it adds no new concept to the existing repertoire,and in itself reveals no property and provides no associatedtheory that was not already available. Its aim and benefit isclarity and focus, achieved by discarding what is irrelevantto the particular purpose in hand.
3.3 Abstraction for formal analysis
By contrast, a more formal kind of abstraction can serve thespecific purpose of demonstrating a desired property of itssubject. This kind of abstraction has been of the greatestimportance in the development of programming, allowingproof that a program satisfies its specification.
In a talk given in Cambridge in 1949, Alan Turing [27]used a flowchart to explain a program and prove it correct.Given n, the program computed n! on a machine withouta hardware multiplier. The flowchart served very well as abridge between the machine-code program and the computa-tion it was intended to perform: it provided a common intelli-
gible abstraction both of the list of machine instructions andof the arithmetic calculation. Turing recognised that the flow-chart abstracted from the machine-code program, observingthat he could not show the ‘routine for this process in full’.That is, he could not show the actual list of machine instruc-tions, because ‘there is no coding system sufficiently gener-ally known’: every computer of the time had its own uniqueorder code. So the flow diagram would have to serve as a sub-stitute: it was a grounded abstraction of the specific patternof machine behaviour evoked by the program.
The elements of the computation were abstracted in theflowchart by a simple notation. Turing showed the allocationof the program’s variables s, r, n, u and v to the machine’sstorage locations 27 through 31, respectively, and he used thevariable names in simple arithmetic operations—for example“r := 1; u := 1” and “test s > r”—in the process and deci-sion nodes. The theme of the talk was program correctness:“How can one check a routine in the sense of making surethat it is right?” With each significant process node of theflowchart, he associated an assertion of the program variablevalues held in each storage location on entry to the node andon exit to each possible successor node. The assertions at theprogram’s entry and exit nodes constituted a program speci-fication in the form of a precondition and postcondition pair:if all assertions on the intermediate nodes hold separately,then when the machine halts this specification is satisfied(Turing used a separate argument to show that the machinemust indeed halt).
Like Beck’s map, Turing’s flowchart provided clarity andfocus, but additionally it provided a basis for formal proof ofthe program’s correctness. In principle, the same proof couldhave been applied to the machine code listing by regardingthe machine state as consisting of its storage locations andregisters together with the program counter. In practice, theproof would then have been considerably complicated bythe distracting intricacies of the machine code instructions.Of course, the flowchart abstraction, while grounded in themachine code program, was not derived from it de novo. Insome form it was no doubt in Turing’s mind, and possiblyalready on paper, when the machine-code program was writ-ten. So while the arithmetic operations can be regarded asvertical abstractions firmly grounded in the correspondingsequences of machine code operations, they may also haveserved as named design elements from which segments ofthe machine code program were refined.
3.4 Multiple horizontal abstractions
The form of horizontal abstraction so effectively practised byHarry Beck becomes more interesting and significant whenmultiple horizontal abstractions are made of the same subjectmatter and must be reconciled in some way. Are the different
123
500 M. Jackson
abstractions compatible? That is: is there a reality of whichthey are all faithful abstractions?
When two abstractions are fully grounded in the sameexistent physical reality, as Beck’s abstractions were, thequestion of their compatibility comes down to the fidelity ofthe abstractions to the reality. If both abstractions are simul-taneously sufficiently faithful to their subject, the physicalexistence of their common physical ground is an incontro-vertible proof of their compatibility. However, compatibilityin the physical reality may be implicitly conditional on aseparation of the contexts of the different abstractions. Boththe horizontal abstractions may be true in the reality, but notat the same time. In this sense, two horizontal abstractionsmay not be compositional. That is: the combination of thetwo abstractions may not possess all the properties of eachabstraction considered in isolation.
Horizontal abstraction can capture a separation of the sub-ject matter into contexts distinguished by purpose and cir-cumstance. An identified context becomes an exclusive focusof attention in its own right. Contexts are not disjoint: somephenomena of the subject matter will be relevant to morethan one context, and common phenomena will play distinctroles in distinct contexts. Program slicing is one example ofhorizontal abstraction.
Another example of horizontal abstraction, drawn fromeveryday life, is the separation of our experience and behav-iour into distinct social contexts. An adult may separate work-ing life from family life; a child may separate school fromhome. Different contexts bring with them different experi-ences, different purposes, and different expectations. To agreat extent, we hope and expect that the different contextswill be neatly separated in time. This temporal separation isvaluable. For practical reasons, and for emotional and intel-lectual comfort, we want to neglect what is irrelevant in thecurrent context: it can only complicate matters. The discom-fort felt by a child—and often not only the child—when par-ents appear at school, or when a teacher from school appearsas a guest at the family dinner table, reflects the difficulties ofsimultaneously managing distinct contexts that the child hadexpected to keep apart. The different roles that the partici-pants are expected to play in the different contexts are hardto reconcile.
The child’s embarrassment precociously demonstrates anawareness of an intellectual problem that besets softwaredevelopment: horizontal abstraction aims to separate con-texts and concerns, but the separation can rarely be perfect.In realistic computer-based systems the horizontal dimensionof abstraction is fundamental, and often there is no temporalseparation. Several horizontal abstractions must coexist intime, because the multiplicity of system features and func-tions, and of normal and exceptional conditions and modesof operation, does not fall into any neat hierarchical tempo-ral structure. The structure of system functionality is more
like a colour separation than it is like an assembly of parts.Horizontal abstractions—albeit imperfectly separated—areessential. We will return to this problem in the followingmain section.
3.5 Program design: top-down and bottom-up
Program design, as widely advocated in the 1960s and1970s, aimed at developing a hierarchical structure ofabstractions to bridge the gap between the computationalabstractions offered by the programming language and theabstractions most clearly associated with the problem to besolved. This gap self-evidently appeared to be a vertical gap:so the abstractions sought were vertical abstractions, eachlayer populated by abstractions of elements of the layer belowit. Naturally, the question arose: should design proceed fromthe top downwards, from the bottom upwards, or by somemixture of the two? Clarifying the distinction more pre-cisely in terms of abstraction: top-down design begins withan abstraction and works to provide successively lower-levelsubjects in which it can be directly and indirectly grounded,while bottom-up begins with the ground and forms suc-cessively higher-level vertical abstractions until a top-level‘solve-the-whole-problem’ abstraction has been constructed.
A kind of consensus arose almost immediately thattop-down design—essentially some form of stepwiserefinement—was the right choice. This consensus wasstrongly influenced by work on the design of small termi-nating programs. The specifications of many of these pro-grams were very simple: the complete specification could betersely and exactly expressed in a precondition and postcon-dition on the program variables. The classic problem corpusincluded: sorting an integer array; finding the greatest com-mon divisor of two integers by Euclid’s algorithm; buildingand maintaining a balanced tree; printing the first thousandprimes; placing eight queens on the chessboard so that noneis attacking any other; finding the convex hull of a set ofpoints in three-space; and other similar problems. Brilliantdesign techniques were invented and described, and somebrilliant solutions were found to problems in the repertoire.
In this setting the top-down consensus has an obviousappeal. The starting point of each program development isone simple abstraction: it is a terse formal specification thatcan form the root of a refinement tree, giving the developera clearly identified anchor at the top end of the vertical gap.Certainly, there may be choices to be made at the early steps,both in formalising the specification and in selecting the solu-tion algorithm; but for a small program, there are usually fewcandidate choices, and once each choice has been made itprovides a firm anchor for the next refinement step.
The bottom end of the vertical gap is furnished by thedefined elements and built-in structures of the programminglanguage. This end is weakly structured with respect to the
123
Aspects of abstraction in software development 501
problem: the eventual need to form an assemblage of pro-gramming language statements rarely constrains the programspecification at the root. Of course, this weak structuring is anadvantage: it springs from, and reflects, the huge versatilityof the programming language and its purposeful flexibility inrealising almost any procedural abstraction. Developers whobegin at the top end of the vertical gap know their startingpoint, and have the programming language readily in mind;they can feel confident that few or no barriers will emergeas they near their goal of an executable program. Developerswho begin at the bottom have apparently more difficulty. Theprogramming language level itself offers little or no obvioushelp in suggesting abstractions that can form the next levelabove; and often it is hard to determine when good progressis being made towards the top-level goal. For the top-downprogrammer the starting point is known, and there are manyacceptable final destinations: for the bottom-up programmerit is harder to know where to start, but only one destinationis acceptable.
Two remarks are in order about top-down and bottom-updesign. First, the physicist Richard Feynman [11], in his per-sonal observations about the Challenger disaster castigatedthe top-down approach to the development of the shuttleengine, and, by implication, to system development in gen-eral. Along with other comments, he wrote “A further disad-vantage of the top-down method is that, if an understandingof a fault is obtained, a simple fix, such as a new shape forthe turbine housing, may be impossible to implement with-out a redesign of the entire engine.” Second, it must be saidthat a rigorous distinction between top-down and bottom-updesign is misconceived. Neither approach is feasible unlessthe designer looks far ahead in the design process, even to theextent of forming a clear mental conception of the completeprogram at the outset. What then proceeds in one directionor the other is the overt written representation of the levels ofthe program as already conceived and of their exact details.
3.6 Program design: horizontal abstractions
The virtues and benefits of top-down development dependcrucially on the simplicity of the problem to be solved, char-acterised by a complete program specification in the form ofone terse simple abstraction. The refinement structure by aprogression of abstractions can then be a tree, in which eachlink between neighbouring steps is in the vertical direction.
When the program specification is not so simple, hori-zontal abstraction becomes inescapable from the outset. Dis-cussing the development of programs from specifications,Burstall and Goguen [12] wrote:
“...the process of implementing a large program fromits specification has a two-dimensional structure. Onedimension of structure, the horizontal, corresponds to
the structure of the specification. The second dimen-sion, the vertical, corresponds to the sequence of suc-cessive refinements of the specification into actualcode; the specification is at the top, and the code isat the bottom.”
The ‘structure of the specification’—the horizontaldimension—arises from the richness of parallel concerns thatmay often be required in a large program. This richness mustbe addressed by multiple horizontal abstractions. Arising inone program, they are unlikely to be disjoint: like the embar-rassed school child, the program designer may find it hardto reconcile them in one coherent design. When horizontalabstraction appears at any refinement node it threatens tocomplicate the whole subhierarchy rooted at that node.
It may be thought that a need for horizontal abstrac-tion rarely appears in a small program, but this is not so.The design of any program that processes sequential inputand output streams—of records, messages, or even singlecharacters—can benefit greatly from appropriate horizontalabstractions. A suitable program-designing method for suchprograms, based on horizontal abstraction, was described bythe present author [17,18]. Each input and output stream isthe subject of a horizontal abstraction in which its sequentialstructure is described as a labelled regular expression. Theprogram structure is then formed by composing these abstrac-tions according to the correspondences among the parts ofthe streams at several levels. If the criteria for a satisfactorycomposition cannot be satisfied the design method invites afurther horizontal composition in which the production ofoutputs from inputs is split into two or more parts interactingby appropriately chosen intermediate streams introduced forthe purpose. This further horizontal abstraction—in effect,a pipe-and-filter design—closely parallels the introductionof intermediate vertical abstraction levels in top-down orbottom-up design.
In computer-based systems, the need for horizontalabstraction is ubiquitous, and is exacerbated there by otherconsiderations peculiar to those systems. We will return tothis topic in the following main section.
3.7 Symbolic and analogical models
Russell Ackoff [2] distinguishes three kinds of model: iconic,analogue and symbolic. An iconic model, as its name sug-gests, represents properties of its subject visually, as in aphotograph or map. An iconic model is primarily a concretething whose properties can be visually inspected—for exam-ple, a portrait sculpture or a three-dimensional map; but itmay also be stored in an intangible form—for example, asthe bit pattern of a digital map or photograph—from whichthe concrete icon can be reconstructed for visual inspection.
123
502 M. Jackson
A symbolic model is abstract: it represents the subject’sproperties in symbols, perhaps in a set of equations, a math-ematical relation, or a logical formula. A formal symbolicmodel is intended to support formal reasoning about the sub-ject. A symbolic model can, of course, be represented con-cretely, but the physical attributes of the representation areof no significance whatsoever. For example, a state machinecan be drawn as a diagram, but the sizes and dispositions ofthe symbols on the page are not significant.
An analogue model is in principle concrete: as its namesuggests, it represents properties of its subject by analogywith its own properties. For example, a hydraulic systemmay serve as an analogue model of an electrical system: thepipes are analogous to wires, and the flow of water to the flowof current. Like an iconic model an analogue model is pri-marily a concrete thing, but may be represented intangibly.For example, an assemblage of programming objects mayconstitute an analogue model, represented in the program-ming language; but, strictly speaking, the assemblage fulfilsits function as an analogue model only when the program isexecuted.
Some models may combine iconic and analogue charac-teristics. Two- dimensional maps, for example, often repre-sent elevation by coloured contours whose gradation fromgreen to red through brown represents increasing elevationby analogy. A photograph necessarily has properties suchas the proportion and relative position of the subject’s fea-tures that are clearly analogues of the reality. For this rea-son, the distinction between iconic and analogue modelsis blurred. It is comparatively unimportant for our purposehere, and we will neglect it, referring to both as analogicalmodels. Symbolic models are more clearly distinguished. Ingeneral, a symbolic model is a pure abstraction of its sub-ject: nothing in a symbolic model need look like the mod-elled subject or be in any other way analogous to it. This iswhy the exact meaning of the model can be preserved undersuitable symbolic manipulations that are useful in reason-ing.
Where the purpose of software is to mechanise calcula-tion, as it was in the earliest days of electronic computers,symbolic models and their known manipulation methods arethe essential basis of program design. The earliest typicalusers were mathematicians and scientists. They understoodprogramming as the task of presenting a symbolic mathemat-ical model to the computer in a form in which the computercould solve the equations and evaluate the formulae of themodel. The first high-level programming language, Fortran,was so named, because it was conceived as a formula trans-lator.
When the purpose of software is expanded to involve amore direct relationship with the human and physical worldit becomes necessary for programs to embody data structuresthat can be used as mutable models, capturing the changing
Interpret-ation S
AbstractSymbolic
Model
SubjectConcrete
Analogical Model
Interpret-ation A
Interpret-ation S
Subject
AbstractSymbolic
Model
Fig. 1 Symbolic and analogical models
states and behaviours of their subject matter more directly.Examples of such mutable models include databases, assem-blages of objects, process networks, and general data struc-tures of the programming language. All these can be analog-ical models.
Unlike a symbolic model, an analogical software modelis not a pure abstraction of its subject. It is, in effect, a mate-rial domain in its own right, albeit a domain enclosed withinthe boundary of the computer. This distinction is important,because the model has its own material properties that aredistinct from those of the modelled domain, and vice versa.The relationship between an analogical model and its subjectis, therefore, more complex than the relationship between asymbolic model and its subject. The two relationships aredepicted in Fig. 1: on the left is a symbolic model and itssubject. The interpretation S maps the terms used in the sym-bolic model to the parts of the subject. In practice, the subjectnames are often used in the abstraction, and the interpretationis then implicit; but in general, an interpretation is requiredto map the terms used in the model to the parts of the sub-ject. On the right is an analogical model and its subject. Theinterpretation S maps the terms used in the symbolic modelto the parts of the subject, while the interpretation A mapsthe same terms to the parts of the analogical model. The anal-ogy is mediated by the common abstraction captured in thesymbolic model
In common software-development practice the symbolicmodel is often—even usually—bypassed, and the analogi-cal model is treated as if it were itself a symbolic modelof the subject. This apparently attractive intellectual short-cut exacts a penalty. As always, the subject has propertiesthat are not captured in the abstract symbolic model; but ananalogical model—unlike a symbolic model—also has itsown concrete properties that are not captured in the symbolicmodel. These properties may correspond, roughly or exactly,to properties of the subject, or they may simply be artifactsnecessitated by the programming language or designed forefficient computation. For example, a database forming ananalogical model may have such properties such as index-ing, ordering of tuples in tables, deletion of inactive tuples,
123
Aspects of abstraction in software development 503
creation of new tuples, and the use of a null value in a fieldwhen no correct value has been entered into the system. Adatabase null value, for example, may represent incompletedata entry; but it may also represent the value undefined in athree-valued logic. When the symbolic model is ignored thesignificance of these additional properties becomes a sourceof difficulty and confusion. The resulting analogical modelsembody ad hoc expedients or demand obscure workarounds“often making it difficult for uninitiated readers of the modelto understand which aspects of the model are meant to beaccurate reflections of the problem domain and which arejust accidental properties of the particular workaround” [3].
3.8 Formal abstractions of a non-formal reality
Formal abstractions provide the most reliable domains of rea-soning that are available to us. Disregarding Gödel, we areable to reason with great confidence in many formal systems,relying on proven theorems to calculate about instances;mathematicians or logicians may also add to the corpus ofproven theorems. We have no fear that someone will one daydiscover that the set of primes is finite, or find some particu-lar triangle in the Euclidean plane the sum of whose interiorangles is not π . Our purpose in constructing formal abstrac-tions of non-formal realities is to impart a similar degree ofconfidence to our reasoning about those realities.
However, there is a severe limitation on the confidencethat can be achieved. A properly constructed formal sys-tem is bounded by its defining set of axioms. But a non-formal reality such as the natural, human and engineeredworld is bounded by no defining set of axioms—at least, notat the scales of interest for practical life in general and soft-ware development in particular. To formalise is inevitablyto place a bound on the phenomena, attributes, relationshipsand possible behaviours that we are prepared to consider.Unfortunately, those that we have oversimplified, ignored orneglected may prove relevant enough to invalidate our for-mal abstraction. More generally, any formal abstraction of anon-formal subject matter is at best an approximation to thereality in which it is grounded. The unpalatable consequenceis that formal reasoning in a non-formal setting can signal thepresence of error but cannot prove its absence. The abstrac-tions that suffice for the premiss may fail for the logicallysound conclusion: it must be tested by interpretation in theground reality.
The history of the traditional engineering branchesabounds in examples of failed abstractions. Leonid Moisseiff,the designer of the Tacoma Narrows bridge, gave carefulconsideration to the possibility of wind-induced horizon-tal oscillation of the roadway; but he neglected the verticaloscillation, which destroyed the bridge a few months after itopened in 1940. The designers of the de Havilland DH106Comet 1 jet airliner were careful to consider all sources of
stress on the aircraft’s fuselage. They made separate andextensive calculations and tests to ensure that the fuselagewould not fail when subjected either to the torsional stressesof flight or to the stresses of the compression and decompres-sion necessitated by the great height at which the aircraft wasdesigned to fly. Unfortunately, it was the untested simultane-ous combination of both sources of stress which caused metalfatigue, spreading from stress singularities sited at the cor-ners of the aircraft’s square windows. The consequent rapidspread of metal fatigue in the structure caused several aircraftto disintegrate in mid-air. In another famous engineering fail-ure in 1978 [15], the roof of the Civic Center in HartfordConnecticut collapsed. The roof structure was a space framedesign that had been formally validated by what was then astate-of-the-art computer analysis. However, the abstractionon which the analysis was based took no account of certainaspects of the proposed design. It ignored the special casesat the outer boundaries of the roof, and also the possibilityof failure by buckling of certain critical members. Worse—and very relevant to software development—when the spaceframe proved hard to assemble on the ground before beinghoisted into its final position, the subcontractor for the roofovercame the difficulty by some seemingly minor changesto the joints between certain members. These minor changeshad an unpredicted major effect on the properties of the struc-ture: the roof collapsed when loaded by an unexceptional fallof snow.
Even in the most modest abstraction there is scope for fail-ure. Harry Beck’s Underground map is an analogical model.It is not itself a formal abstraction, but nonetheless it illus-trates some of the difficulties of even a partial formalisation ofa non-formal reality. It seems obvious that the Undergroundsystem consists of stations connected by tracks. But imme-diately the notion of station confronts us with a difficulty.What, exactly, is a station? At first sight Edgware Road is astation; but on closer inspection it appears to be two stations:to change trains there requires leaving the Underground sys-tem, walking 150 yards along the street and re-entering theUnderground system at ‘the other Edgware Road’. Bank andMonument are distinct named stations; but they have beenlinked into one since they were originally built in the nine-teenth century.
Anomalies and exceptions of this kind are characteristic ofnon-formal realities. They arise from the interpretation of for-mal terms, which are the very foundation of any abstraction:what counts as a station? They arise also from peculiar cor-ner cases of reality that present counterexamples to almostany satisfyingly elegant theory associated with the chosenabstraction. They arise from the difficulties of constructinga physical reality to conform faithfully to its design abstrac-tion. They arise from the untoward combination in reality oftwo horizontal abstractions such as the two stress analysesof the Comet aircraft.
123
504 M. Jackson
These difficulties arise for an engineered system like theLondon Underground, and even more strongly for the com-plex and arbitrary organisational, commercial, manufactur-ing, legal, and fiscal realities that make up the problem worldfor many large systems. In a non-formal world, clean abstrac-tion for a purpose presents a discouraging dilemma. Thecleaner the abstraction the more anomalies will emerge toreduce its fidelity to the reality from which it sprang andwhich it purports to describe.
3.9 Abstractions and monsters
Anomalies and counterexamples, characteristic of abstrac-tion in non-formal domains, may be encountered also indealing with a formal mathematical domain. Lakatos [23]recounts the mathematical history of Euler’s formula V −E+F = 2 relating the numbers of vertices, edges and faces of apolyhedron. He introduces a series of solid figures, proposedas counterexamples to the conjectured theorem implicit in theformula. Successively he introduces: a hollow cube whoseinterior faces form a smaller cube; a ‘crested cube’ having asmaller cube projecting from the middle of one of its faces;a pair of tetrahedra sharing only one edge; a pair of tetrahe-dra sharing only one vertex; a ‘star polyhedron’, and severalothers.
A central theme in the discussion is ‘monster-barring’.Some mathematicians, to preserve the conjecture, rejectedputative counterexamples as ‘monsters’—solid figures, butnot polyhedra. Challenged to define polyhedron, theydefended the formula by producing over time a sequenceof successive definitions carefully modified to exclude eachsuccessive proposed monster. With the hindsight of the his-tory of Euler’s formula, we may fault the mathematiciansfor being insufficiently careful in formalising the polyhe-dron abstraction. But in dealing with the phenomena of thephysical and human world at the granularity of interest toengineers and software engineers, the existence of monsters,challenging any proposed abstraction and its associated the-ory, is undeniable: for every definition there are hard casessomewhere in the world; and for every theorem there arecounterexamples.
How, then, can we proceed in practice if our abstractionsand theories are so fragile? How can we avoid the monstersthat lie in wait for us? We can adopt various strategies accord-ing to our purposes and circumstances. For example:
• We may simply accept some degree of approximation anda consequent level of imperfection and even failure. Theanomalies of certain stations will inconvenience someUnderground passengers; but not many passengers, andnot often.
• We may rethink or elaborate the abstraction to preservethe associated theory, in the spirit of the successive def-initions of polyhedron. The efforts of mathematicians topreserve Euler’s conjecture against the challenge of suc-cessive monsters find some echo in the energetic effortsof engineers to eliminate the risks of successively iden-tified sources of failure in such safety-critical artifacts ascars, power stations and aeroplanes. National tax author-ities work hard to close the successive fiscal loopholesfound by ingenious lawyers and accountants.
• We may supplement the abstraction and the implementa-tion of the theory by allowing an overriding human judg-ment for the hard cases. This is the usual practice in a goodlegal system. The law relies on abstractions of many kindsof human behaviour, and stipulates rules, rights, obliga-tions, and penalties in terms of those abstractions. Whenan abstraction or theory breaks down because it is notclear how it should apply to the facts of a particular case,the matter is settled by the judgment of a court.
• We may apply the abstraction and the accompanying the-ory only in a strictly limited context in which we are con-fident that no monsters are to be found. In the context ofall human life the commonplace abstraction chair is veryproblematical. Is a bar stool a chair? A bean bag? Theseat of a swing? A bicycle saddle? A park bench? TheT-bar of a ski lift? But in the context of the sales systemof a furniture factory with a narrow product range it maywork perfectly.
Strategies of these kinds are essential in the devel-opment of computer-based systems. The last strategy—context restriction—is particularly important: developers ofa computer-based system must not aim at an unattainable andpointless universality.
4 Abstraction in computer-based systems
In a computer-based system the computer monitors and con-trols parts of the human, physical and engineered world out-side itself, evoking and imposing some required behavioursin that world. This is its purpose: its own internal behaviour,and its participatory behaviour in its direct interactions withthe world, are merely the means to that end [19].
Developers of such systems cannot restrict their focus tothe clean and formal computational structures quarantinedwithin the bounds of aseptic program texts executed by amachine that reliably implements a formal instruction set. Insuch developments, abstraction remains a vital intellectualtool; but its power is comparatively diminished in relation tothe overall task. Of course, most realistic system- develop-ment projects involve subsidiary tasks of formal specification
123
Aspects of abstraction in software development 505
and program design; but these tasks are overshadowed bylarger concerns. The exercise of abstraction in system devel-opment demands more judgment, more scepticism, moreinsight, more versatility, and more hard work. For exam-ple, the non-formal nature of the problem world calls almostevery proposed abstraction into question. In a formal world,after the instruction sequence
x := P; y := x; x := y, (1)
the condition “x = P” will certainly hold. But in the materialworld, after the instruction sequence
x := P; Arm.moveT o(x);x := Arm.current Posi tion,
(2)
the condition “x = P” may not hold. Moving the arm andsensing its position both involve state changes in the problemworld. Movement of the arm will be imprecise, and the result-ing position will be imperfectly sensed by the computer andfurther approximated by a floating-point number. This kindof difficulty is exacerbated by the multitude of potentiallyconflicting system requirements to be satisfied by a realisticsystem and by the further complexities they add at almostevery level.
This section discusses some aspects of the use ofabstraction in the particular context of computer-based sys-tem development. First some of the salient characteristics ofcomputer-based systems are identified and briefly discussed.Then, in the remaining subsections, some of their implica-tions for the use of abstraction are explored.
4.1 Characteristics of computer-based systems
Computer-based systems are very various. Few characteris-tics are common to them all, and few of them exhibit allthe characteristics identified here; but wherever these char-acteristics are found they are likely to complicate the use ofabstraction in system development in some way and diminishits utility.
Many people and organisations—the ‘stakeholders’ inthe accepted jargon of requirements engineering—may bein some way involved in the system operation: their vari-ous needs must therefore contribute to determine the sys-tem requirements. For example, a radiation therapy systeminvolves therapists, patients, oncologists, radiologists, med-ical physicists and maintenance engineers: the behaviour ofthe system must allow each of them to play their part suc-cessfully. If the system is safety-critical, a safety authorityis also a stakeholder, demanding that the developers showconvincingly that the system is acceptably safe.
The problem world of the system is likely to be a hetero-geneous assemblage of problem domains. Some are mecha-tronic devices engineered to published specifications. Some
are human participants, ranging from carefully selected andhighly trained operators such as aircraft pilots and traindrivers to uninformed customers, randomly selected casualusers, and medical patients. Some are parts of the natural orbuilt environment such as airport runways, a tanker terminal,the earth’s atmosphere at various elevations, or a network ofrailway tracks. Some are existing systems such as the inter-net, the telephone system or the global positioning system.Each problem domain has its own given properties, whichthe system must respect and exploit and the developers musttherefore understand and analyse.
One system has many features. For example, an auto-motive system may have driving assistance features such aselectronic suspension control, start–stop, anti-skid braking,cruise control, maximum speed regulation, lane departurewarning, and automatic parking. These features can interactby requirement conflict: in some circumstances, two featuresmay require contradictory behaviours. They can also inter-act through common problem domains that are not explic-itly mentioned in the individual features’ requirements: forexample, by imposing excessive demands on engine or bat-tery power. In general, the relationship between problemdomains and the behavioural features in which they partici-pate is many-to-many.
The system may have multiple modes of operationdemanding different functional behaviour. The behavioursof a radiotherapy system must include, for example: acquir-ing and validating an oncologist’s prescription for a patient;determining the position of the patient for a first treatmentaccording to the prescription; repeating a previously deter-mined position for a subsequent treatment; managing theradiation dose in a treatment; initial setup and calibration;and operation under control of a maintenance engineer.
Inescapably, any system is designed to function in arestricted context that affords certain assumptions. For amotor car, for example, explicit context restrictions mayspecify such obvious factors as fuel, ambient temperature,tyre pressures, regular oil changes, and so on. Other contextrestrictions may be implicit: few car manuals state explicitlythat the car will not function under water or on the moon,cannot be satisfactorily driven on sand dunes, will not climba one-in-one gradient, and will not carry a load of ten tons.Many context restrictions are left to be understood by com-mon sense, or expressed in such phrases as ‘normal use’.
Context is important because both the requirements andthe given problem domain properties depend, in general, onthe context of system operation. Natural phenomena maysometimes exhibit unexpected behaviour such as a gale or atsunami. Engineered devices may wear out and cease to func-tion as specified. The assumed bounds of human operators’behaviour may be exceeded—for example, input speed mayincrease with long familiarity with the system. The maximumpermitted length of a railway train may increase over time,
123
506 M. Jackson
changing its relationship to the length of a track segment. Dif-ferent contexts are likely to demand different abstractions.
The operational context and its accompanying assump-tions are not global to the system. A critical system must bedesigned to take proper account of variations in context: theoperational context at any point in time is a set of overlappingcurrent subcontexts. Fault-tolerance is merely one particularexample: the fault presents a subcontext for which the sys-tem has a specified behaviour, perhaps providing a degradedfunctionality or shutting the system down completely accord-ing to the severity of the fault. For a passenger lift system,the presence of a fire in the building presents another subcon-text: the system must provide a form of lift service specif-ically designed for the needs of the firefighters. Obviously,the fault and fire subcontexts are not mutually exclusive. Anautomotive system must behave in different ways when thecar is in normal use, when it has been involved in a collision,and when it is in a repair workshop. The more critical the sys-tem the more various and extreme the subcontexts in whichit is required to behave in specified safe ways. The designof a nuclear power plant was recently criticised for failingwhen a tsunami and an earthquake of magnitude 8.9—bothrare events—occurred simultaneously.
4.2 Abstractions and purposes
We make an abstraction to serve a purpose, and its value andsuccess are to be judged by how well they serve that purpose.Developing a computer-based system is a task in which manydifferent purposes must be pursued, and it follows that manydifferent abstractions will be needed. This profusion of pur-poses is, in general, not found in the development of smallprograms whose subject matter is a mathematical abstrac-tion. For these programs the given properties of the subjectmatter—or problem world—are well known, or can be reli-ably learned by consulting a mathematical text. The devel-oper can exploit different aspects of the given properties byrelying on a corpus of proven theorems. Essentially, the onlyabstraction to be invented is an abstraction of the programbehaviour in terms suited to the chosen programming lan-guage.
For a computer-based system, by contrast, it is necessaryto capture, at a suitable level of abstraction, the relevant givenproperties of each problem domain, perhaps as they vary withthe operational context. Even where a problem domain is adevice engineered to an explicit specification, there will stillbe a need to identify the properties on which the system canrely in each of its various operational contexts and modes.This is, evidently, a task of abstracting for description. Insome cases the task may be essentially one of selection, pre-senting itself as horizontal abstraction. For example, it maybe very useful to make separate abstractions of faulty andfault-free behaviour: the faulty behaviour is significant for
fault detection and diagnosis, while the fault-free behaviouris significant for normal operation. In other cases descriptionmay involve vertical abstraction, abstracting and analysing ahigher-level behavioural property from concrete lower-levelbehaviour. In neither case can the abstraction be adequatelytreated top-down—that is, as a task of refinement rather thanabstraction. Refinement is a process of inventing and con-structing something new, not a process of describing an givenexisting reality.
Invention and construction is the process for develop-ing the system’s behaviour—or, more properly—its manybehaviours. Once a sufficient set of given problem domainproperties has been captured by a process of abstraction, itmay become practicable to devise a desired system behaviourfrom the top-down, just as it may be possible to design a pro-gram from the top-down once the programming-languageelements are known. However, there is a crucial side con-dition. Strict top-down development is feasible only whenthe required behaviour can be tersely specified to define thegoal of the first refinement step. For the overall behaviourof a computer-based system this is rarely—perhaps never—possible. The overall behaviour is an assemblage of severalfunctional behaviours that must come into play in responseto changing operational circumstances and potentially unpre-dictable user demands. This overall behaviour as a whole can-not be usefully abstracted to give an effective starting pointfor refinement. Instead the individual functional behavioursmay be separately designed, each taking explicit account ofthe subcontexts in which it is required. Recombining theseseparately designed behaviours becomes a further develop-ment task.
Third purpose, along with description and construc-tion, is analysis. Given an existing reality—whether theresult of description or of construction—analysis makesan abstraction with the purpose of validating a claim thatthe reality possesses some desired property or exhibitssome desired behaviour. If the abstraction and the vali-dation process are formal the validation may proceed byproof or model-checking, and is then usually called veri-fication, the term verification connoting a degree of confi-dence associated with a mathematical demonstration. Theuse of this term is fully appropriate for the mathemati-cal demonstration itself; but, of course, it is quite inap-propriate to the question whether the formal abstractioncorresponds faithfully to the reality in which it is ulti-mately grounded and in which the purpose of the systemis located.
The fourth purpose arises from the multifarious natureof the stakeholders and their requirements. In a computer-based system, many requirements are characterised as ‘non-functional’. A notable example is usability. This requirementis, in fact, purely functional in the sense that its satisfac-tion or non-satisfaction can be judged by observing the
123
Aspects of abstraction in software development 507
functional behaviour of the system—including, of course,its users, operators and other human participants. Althoughcognitive and ergonomic research has more to say aboutusability, the judgment still cannot be made by the devel-opers themselves alone: the stakeholders or their legitimaterepresentatives must play a decisive part. When develop-ers design the whole system behaviour they must therefore,make abstractions of that behaviour that capture the asso-ciated participating behaviours of humans in their variousroles. The stakeholders must validate their proposed par-ticipation by their assent based on a full comprehension.According to the criticality of the system, and the nature ofthe participation and the stakeholders involved, this compre-hension may be achieved by examining a symbolic abstrac-tion such as a state machine, by viewing an animation, byinteracting with a prototype implementation, or by othermeans.
For a realistic computer-based system, the multiplicity ofpurposes which abstraction can serve, together with the rich-ness of the system functionality, properties and behaviours,makes it clear that many abstractions are necessary to sup-port and embody the development process. In the followingtwo subsections, two particular abstractions, each an exam-ple of a widely used class, are discussed. The purpose of eachone is briefly explained and some of its virtues and limita-tions are identified. Advocates of each will no doubt be ableto enlarge the list of its virtues. The limitations, it must beunderstood, are not presented here as culpable defects: theymerely emphasise the truth that one abstraction alone cannotsuffice.
4.3 An example abstraction: Event-B
The Event-B refinement method [1] is based on a formalabstraction of system behaviour. The system has a globalstate that is modified by events. It shares this fundamen-tal abstraction with other development methods, includingZ [30] and VDM [20]. Events have arguments denoting ele-ments of the state, and are guarded by predicates on the state.Some predicates on the state are defined as invariants thathold in every state. The system is consistent if all invariantshold for all possible sequences of events. The purpose of thisabstraction is to support a development discipline. Startingfrom a very abstract model, capturing an initial understandingof the problem domains and the requirements to be satisfied,the development proceeds by successive refinement steps. Ineach step a more detailed model is constructed and proved tobe a refinement of the preceding model: that is, while addingdetail, it preserves the invariants and other properties of themore abstract model.
Formal reasoning with this abstraction is very tractable. Toprove that a model is consistent and refines its more abstractpredecessor it is necessary to complete many small proofs,
not all of them are trivial. The chosen abstraction of sys-tem behaviour often allows most of these proofs to be per-formed automatically by specialised software tools, leavingrelatively few proofs to be devised and carried through byhand. This is a large benefit.
Any abstraction has two faces: what is included, andwhat is discarded. The Event-B abstraction discards manyphenomena and considerations that are significant for somedevelopment purposes. In particular:
• Different parts or domains of the system are not distin-guished. No distinction is made between events occur-ring in the computer and events occurring in a problemdomain, or between the computer’s internal states andinternal states of the problem world. These are importantdistinctions in the practical utility of the system. Withoutthem, it is, for example, impossible to address a possibledivergence between a problem domain state and the stateof its analogical model in the computer.
• Causality is ignored. It appears in the model only in theassociation of an action with an event: each occurrenceof the event may be imagined to cause the action. Sincethe initiator of the action is not identified, this associationdoes not capture causal relationships. Such relationshipsare essential. To understand how the system works, andto demonstrate that it will work reliably, it is necessaryto trace the causal chains that define its proper working,and to consider the possibilities of failure in each link ofeach chain.
• An invariant may represent a requirement—for example,“an employee is never in a room for which the employeedoes not hold an access authorisation”; or it may representa given or assumed property of a problem domain; forexample, “no pair of rooms is connected by more than onedoor,” or “a train can move out of a track segment only toan adjacent segment”. A requirement can be modified byagreement with the stakeholder; a given domain propertycan be modified only by a change in the physical world.
• The context of system behaviour is assumed to be uni-form. However, different behaviours are required in dif-ferent contexts. For example, “two trains never occupythe same track segment” is true for train journeys butfalse when a train is being assembled in preparation for ajourney or has broken down and is to be towed to a repairshop. The distinction between different contexts can berepresented only by adding the context as a state elementand conjoining a predicate on its value to the affectedinvariants and event guards. This representation wouldbe very cumbersome and error-prone.
• Sequential behaviours cannot be directly represented inEvent-B. Sequencing can be captured only by a relation-ship between the changing system state and the event
123
508 M. Jackson
guards. Sometimes, this relationship can be defined interms of state elements clearly associated with problemworld states; sometimes it requires the introduction of avariable that is, in effect, a partial representation of thetext pointer of the sequential process. This very indirectad hoc approach fragments a sequential behaviour andmay destroy its unity and human intelligibility.
4.4 An example abstraction: use cases
A completely different, informal, abstraction of systembehaviour is implicit in the widely practised technique of usecases [22]. A use case is an episode of interaction betweenan actor—typically a human user—and the system; for thepurposes of the use case the actor is regarded not as a part ofthe system but as an external agent. The episode of interac-tion delivers some result of value to the actor; for example,the actor succeeds in booking a theatre seat, or in drawingcash from an ATM. A use case is described informally as aninteractive sequential process. The process may have manyvariations. For example, the theatre may be fully booked;the user may decide not to accept any of the available seats;the ATM cash may be exhausted; the user may fail to enterthe correct PIN for the card inserted; the user’s account bal-ance may be insufficient; the process may time out, and soon. To accommodate common subprocesses such as logginginto the system, validating the card inserted, or paying bycredit card, use cases may be structured to embody or invokeother use cases.
Use cases are often understood as the central—sometimesthe only—vehicle for describing required system behaviour.Philippe Kruchten [22] writes:
“The use-case model is a model of the system’s intendedfunctions and its environment, and it serves as a contractbetween the customer and the developers. It comprises theset of all use cases for the system, together with the set of allactors, so that all functionality of the system is covered.”
The value of use cases is obvious. They describe the expe-riences that the system must afford to its users when theyavail themselves of its various user-initiated functions, andallow the developers to design those experiences for users’convenience and satisfaction.
The use-case abstraction, like an Event-B model, discardssome significant phenomena and considerations:
• Some system behaviour is evoked not by immediate userinteraction but by a change in system state; for exam-ple, by a change in the relationship between outstandingorders and stock-in-hand for a product. Such behavioursare not easily or fruitfully described in terms of deliveringa result of value to a user.
• The role of user does not accommodate other importantroles that a person may play. For example, the behaviourand needs of the recipient of a heart pacemaker are notexactly those of a user. Rather, the recipient’s cardiacbehaviour is the subject of monitoring and control bythe embedded computer: the recipient is scarcely morea ‘user’ of the pacemaker than the patient in a surgicaloperation is a ‘user’ of the operation theatre.
• The fragmentation of user behaviour into use-caseepisodes works well when each use case can be regardedas an independent episode rather than as a contributionto a larger purpose that persists across distinct use-caseinstances. This assumption largely holds for a telephonesystem, in which each use case can be largely understoodin isolation; but it does not hold for the driver-assistancefunctionality of a car or for patient treatment by a radia-tion therapy machine.
Discarding significant aspects of a problem is not in itself afault. On the contrary, it is essential to separation of concerns.It becomes a fault only when the abstraction in question isregarded as the only abstraction necessary for development.
4.5 Representation and comprehension
The representation chosen for an abstraction plays a largepart in its comprehensibility. Since programming and sys-tem development are essentially human intellectual activ-ities, they can be carried out most effectively when theircontent is thoroughly understood by the people involved.The developers must understand the abstractions they con-struct, and the stakeholders must understand the content ofthe abstractions to which they are asked to assent. An abstrac-tion can be represented in more than one way. Whether it iscomprehensible depends not only on its formal content butalso—vitally—on its representation.
To take a well-worn example, a state machine can be rep-resented by a diagram or, equivalently, by a list of nodes andtransitions. The fragmented list is well suited to process bycomputer, but the diagram is absolutely essential for humancomprehensibility. The distinction runs deep. As the mathe-matician Henri Poincaré [25] wrote:
“When the logician has resolved each demonstrationinto a host of elementary operations, all of them correct,he will not yet be in possession of the whole reality;that indefinable something that constitutes the unity ofthe demonstration will still escape him completely.”
It is a major misfortune for software and system develop-ment that a fragmentary form of an abstraction—alist of nodes and transitions, or a collection of elemen-tary operations—is usually more tractable by mechanised
123
Aspects of abstraction in software development 509
processing. The danger is that as formal analysis by theoremprovers becomes more powerful, and therefore more attrac-tive, it leads to a weakening of the crucial demand for humanunderstanding. Sequential processes are a fundamental partof human experience in the world, and we have all learned tograsp that indefinable something that they convey. A processrepresented by fragments ceases to be humanly comprehen-sible: the links between fragments formed by state variablesare no substitute for a coherent representation of the wholeprocess. Dijkstra [5] explained their inadequacy in his famousletter about the structure of a program text:
“The reason is—and this seems to be inherent tosequential processes—that we can interpret the valueof a variable only with respect to the progress of theprocess.”
To support both human comprehension and machinetractability, more than one representation may be necessaryfor the same abstraction. Effective development support soft-ware must at least be capable of deriving the comprehensiblerepresentation from a more machine-tractable equivalent.
4.6 Abstraction by context
A major feature of a realistic computer-based system isits multiplicity of operational subcontexts. An aircraft mustbehave differently in the different phases of a flight: stand-ing, pushback, taxiing, takeoff, climbing, en route, approach-ing, landing, and so on. A car must behave differently onthe highway and in the repair shop. A lift in a large multi-purpose building must behave differently when the equip-ment is functioning perfectly and when the equipment isfaulty, differently in the morning and evening, and differentlyat the weekend and on weekdays. In these different contextsthe system requirements will be different, and so also willthe envelope of given properties of the problem domains;for example, the aircraft engines, the car suspension, the liftusers’ behaviour.
Horizontal abstraction is the necessary tool for separatingthe different subcontexts. In the absence of this separation,the design of any particular functional behaviour can relyonly on the weakest assumptions. For example, the designand provision of normal lift service behaviour must rely onproperly functioning equipment. The developer who inte-grates into this design the detection, diagnosis, and handlingof equipment faults is addressing a problem that is very com-plex for reliable solution. Almost nothing can be assumed,so at every point in the designed behaviour it is necessaryto check which of a very large number of possible statescurrently holds. Eventually the reflective developer will findit desirable to structure this complexity by introducing addi-tional state variables: Has a fault already been detected? What
fault? Is the system already trying to recover from a fault?Has normal lift service already been abandoned? Is normallift service currently in course of being abandoned by bring-ing the lift to a safe floor? This necessary structuring of sys-tem state is exactly the structuring aimed at by a horizontalabstraction by context.
Separation of concerns is a generally recognised principlein the mastery of complexity. Less generally recognised isthe need to recombine the separated concerns to produce asatisfactory overall behaviour. Sometimes, in a small non-critical setting, this recombination can be almost completelyavoided: the designed behaviour is aborted and the system,or the affected part of it, will be restarted later from a care-fully specified initial state. For example, in the classic usecase of withdrawing cash from an ATM there are many pos-sible ways of failing: the user’s card may be faulty; the PINmay be wrong the card may have been previously reportedstolen, and so on. In the case of such a failure the card may beretained in the machine and the use case is aborted. Dealingwith the retained card and with the possible explanations forPIN errors are system behaviours that need not be tightly inte-grated into the ATM use case but can be dealt with elsewhereas a separate behaviour for a separate context. However, manysystems, including some of the most critical, are requiredto operate continuously. Aborting the current behaviour andrestarting elsewhere from a carefully specified initial stateis not a permissible design choice in automotive or avionicsystems. Recombining behaviours separated by horizontalabstractions then presents various challenges.
Here, we will mention two recombination challenges.First, when horizontally abstracted behaviours can overlapin time, one of them may be based on the assumption thatthe relevant properties of a shared problem domain will beunaffected by the other. For example, the scheduling of trainservices may be separated from the scheduling and manage-ment of track maintenance, relying on the assumption thatthe separation can be perfect. At any time, the rail networkcan be partitioned into those tracks on which services can bescheduled and those on which maintenance work can be per-formed. This assumption may be false. The recombination ofthe two behaviours then must take their mutual interferenceinto account, modifying one or both of them accordingly. Thetwo horizontal abstractions are not compositional.
Second, when horizontally abstracted behaviours areconsecutive in time, it is necessary to consider whether theproblem world post-state of the earlier satisfies the assumedproblem world pre-state of the later. For example, normal liftservice and firefighter lift service may be consecutive: duringnormal lift service a fire is detected and the system must beplaced under control of the fire brigade. But at the momentof detection the lift car may contain passengers, and may beengaged in a journey to satisfy their requests and other pend-ing floor requests. It will be necessary to design what may be
123
510 M. Jackson
called a switching behaviour to deposit any passengers at asafe floor before handing over the lift to fire brigade control.
Horizontal abstraction by context is not, of course,restricted to computer-based systems. A very different exam-ple is seen in the parsing of an input text that may containsyntactic errors. The text is a problem domain, and its outerboundary of properties in the overall context is constrainedby the input mechanism: for example, the character set maybe constrained by a keyboard. In one horizontal abstraction, asyntactically faultless text is assumed, having a well-definedstructure of which the parser takes advantage. An element inthis structure may be white space, an abstract lexical tokenin which any unbroken sequence of space, tab, and carriagereturn characters is equivalent to any other. For the mosthelpful diagnosis of errors, however, it may be important toadopt a different abstraction. If the constituent phenomena ofwhite space are not discarded, the physical layout of the textlines can be explicitly recognised: mistyping of a right braceis then more easily pinpointed and diagnosed in a carefullyindented text.
5 Carefully thoughtful use of abstraction
The variety of software-development problems is huge, andkeeps growing. Some classes of system have evolved effec-tive standard designs and development procedures, but manyhave not. Much software development is, therefore, not a rou-tine activity: it comprises a high proportion of radical, ratherthan normal, design [28]. The developer is to that extentunable to rely on a standard designs evolved and validatedby many practitioners over a long period, and must fall backon a personal capacity for invention. This pleasurable inno-vative activity must be accompanied by a strong inclinationto self-questioning. From the first investigation of require-ments through to system testing and installation, develop-ers can benefit from questioning what they are doing: fromconsidering explicitly what abstractions they are using, ques-tioning the nature of those abstractions, and articulating howthey are related to the purposes of their work and the realitiesin which they are ultimately grounded.
When difficulty is encountered, it is always good to ques-tion the abstraction or set of abstractions within which thedifficulty has arisen. Advocates of aspect-oriented softwaredevelopment speak of “the tyranny of the dominant decom-position”, and the need to escape, somehow, from the strait-jacket it imposes. In the same spirit we may speak of thetyranny of the dominant abstraction. An abstraction thatserves well for one purpose can easily become ‘sticky’: webecome unable to escape when for other purposes it becomesa tarpit.
A famous historical example is the Pythagoreans’ dom-inant abstraction of numbers: all numbers are rational.
According to the tradition, the discovery that the square rootof 2 is irrational was more than they could bear; it is even saidthat they murdered Hippasus, its discoverer. Another exam-ple, more obviously germane to software development, is theidea of a telephone call. A call is an attempt by one telephone(the caller) to establish one connection to one other telephone(the callee). When it became apparent in the 1990s that tele-phone systems were becoming more powerful and user fea-tures were proliferating, an international effort developed astandard conceptual model for telephony which was entirelybased on the call abstraction. Unfortunately, many telephonefeatures subvert the one-to-one correspondence that is theessence of this abstraction [31]. Conference calls, voicemail,automatic callback, credit-card calling and many other fea-tures simply cannot be clearly described on the basis of thecall abstraction.
Simplification is an important use of abstraction. HarryBeck’s map simplified the task of planning a route onthe London Underground. It also provides an object les-son in one aspect of the dangers of an analogical model.Although in principle Beck had discarded the geographicalphenomena, top to bottom of his map was still, roughly, northto south, and left to right was still west to east. His abstractionwas therefore partial, in the sense that the geographical phe-nomena were only partially discarded: geographical locationstill influenced the positions of stations on the map, but imper-fectly and inconsistently. Inevitably, such a partial abstractionis potentially misleading: some users wrongly suppose thatdistances are exactly preserved. In one extreme case, a usermay undertake a journey visiting four stations and changingbetween two Underground lines to travel between two sta-tions that are 250 yards apart. Both for the maker and for theuser of an abstraction it is vital to understand clearly exactlywhat has been discarded. In the practice of abstraction, thebaby is not always easily distinguished from the bathwater.
Acknowledgments I am grateful to Daniel Jackson for illuminatingdiscussions. I also thank the anonymous reviewers, whose careful andextensive comments and suggestions have encouraged and helped meto improve this essay.
References
1. Abrial, J.-R.: Modeling in Event-B: System and Software Engi-neering. Cambridge University Press, Cambridge (2010)
5. Dijkstra, E.W.: A case against the GO TO statement: EWD215.Commun. ACM. 11(3), 147–148 (1968). published as a letter tothe Editor
123
Aspects of abstraction in software development 511
6. Dijkstra, E.W.: The structure of the ‘THE’ multiprogramming sys-tem: EWD196. Commun. ACM. 11(5), 341–346 (1968)
7. Dijkstra, E.W.: The Humble Programmer. Turing Award Lecture.Commun. ACM. 15(10), 859–866 (1972)
8. Dijkstra, E.W.: A Discipline of Programming. Prentice-Hall, USA(1976)
9. Dijkstra, E.W.: On the cruelty of really teaching computer science.Commun. ACM. 32(12), 1398–1414 (1989). With responses fromDavid Parnas, W L Scherlis, M H van Emden, Jacques Cohen, RW Hamming, Richard M Karp and Terry Winograd, and a replyfrom Dijkstra
10. Evans-Pritchard, E.E.: Witchcraft, Oracles and Magic among theAzande. The Clarendon Press, Oxford (1937). (Abridged with anintroduction by Eva Gillies. The Clarendon Press, Oxford (1976))
11. Feynman, R.P.: Personal observations on the reliability of theShuttle; Appendix F to the Rogers Commission Report, (1986);available at http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/rogers-commission/Appendix-F.txt, accessed 19th (May2012)
12. Goguen, J.A., Burstall, R.M.: Cat, a System for the StructuredElaboration of Correct Programs from Structured Specifications:Technical Report CSL-118. Computer Science Laboratory, SRIInternational, USA (1980)
13. Gregory, R.L.: The medawar lecture 2001 lnowledge for vision:vision for knowledge. Philos. Trans. Royal Soc. B. 360, 1231–1251(2005)
14. Hardy, G.H.: A Mathematician’s Apology. Cambridge UniversityPress, Cambridge (1940)
15. http://matdl.org/failurecases/Building_Collapse-Cases/Hartford_Civic_Center, accessed 3 October 2011
16. Jackson, D.: Software Abstractions: Logic, Language and Analy-sis. MIT Press, USA (2006)
17. Jackson, M.A.: Principles of Program Design. Academic Press,London (1975)
18. Jackson, M.A.: Constructive methods of program design. In: Goos,G., Hartmanis, J. (eds.) Proceedings of the 1st Conference of theEuropean Cooperation in Informatics, p. 262. Springer, Berlin(1976)
19. Jackson, M.: Some Basic Tenets of Description. Softw. Syst. J.1(1), 5–9 (2002)
20. Jones, C.: Systematic Software Development Using VDM, 2ndEdition. Prentice-Hall International, USA (1990)
21. Jeff, K.: Is abstraction the key to computing? Commun. ACM.50(4), 37–42 (2007)
22. Kruchten, P.: The Rational Unified Process: An Introduction.Addison-Wesley Longman, Reading (1999)
23. Lakatos, I.: Proofs and refutations. In: Worrall, J., Zahar, E. (eds.)The Logic of Mathematical. Cambridge University Press, Cam-bridge (1976)
24. Parnas, D.L.: On the criteria to be used in decomposing systemsinto modules. Commun. ACM. 15(12), 1053–1058 (1972)
25. Poincaré, H.: Science et Méthode. Flammarion, France (1908)(translated by Francis Maitland, Nelson 1914, Dover, 2003)
26. Billson, C.: A History of the London Tube Maps. http://homepage.ntlworld.com/clivebillson/tube/tube.html. accessed 30 May 2011
27. Turing, A.M.: Checking a large routine. In: Report on a Conferenceon High Speed Automatic Calculating Machines, pp. 67–69, Cam-bridge University Mathematical Laboratory, Cambridge, (1949).(Turing’s paper is discussed in Cliff B. Jones; The Early Search forTractable Ways of Reasoning about Programs. IEEE Annals of theHistory of Computing 25(2), 26–49, 2003)
28. Vincenti, W.G.: What Engineers Know and How They Know It:Analytical Studies from Aeronautical History. The Johns HopkinsUniversity Press, Baltimore (1993)
29. Hermann, W.: David Hilbert and his mathematical work. Bull. Am.Math. Soc. 50, 612–654 (1944)
30. Woodcock, J., Davies, J.: Using Z: Specification, Refinement, andProof. Prentice-Hall International, USA (1996)
31. Zave, P.: Calls considered harmful and other selected papers onservices and visualization. In: Tiziana, M., Bernhard, S., Roland,R., Joachim, P. (eds.) Towards User-Friendly Design, LNCS 1385,p. 27. Springer, Berlin (1998)
Author Biography
Michael Jackson has workedin software since 1961. Hisprogram-design method, descri-bed in Principles of ProgramDesign (1975), was chosen asthe standard method for UK gov-ernment software development.Later work at AT&T, on telecom-munication systems architecture,is the subject of several patents.His work on problem structureand analysis is described in Soft-ware Requirements & Specifica-tions (1995), in Problem Frames(2001), and in many published
papers. He has visiting posts at The Open University and the Univer-sity of Newcastle, participating in research projects there and at otherresearch and academic institutions. He has received several researchawards, including the British Computer Society Lovelace Medal, theIEE Achievement Medal, and the ACM Sigsoft Outstanding ResearchAward.
