A software packagespecification validation processby S. D. Fraser and Prof. P. P. SilvesterMcGill University

This paper proposes a technique for specification validation, termedthe scenario-based prototype approach. This method of validation isparticularly useful for software packages comprising substantialnumbers of unrelated or loosely coupled functions able to act on aunified database but not restricted as to their actions or sequencing.Such packages commonly occur in computer-aided design systems,numerical mathematics software and graphics displays. The proposedmethodology has been implemented using the INGRES databasemanagement system. It has been used to validate the specification ofthe Graphics Kernel System and to simulate the operation of GKS atlevel 0.

Introduction

Well designed software systems forcomputer-aided engineering or designmay be viewed as special-purpose datamanagement systems. They generallyinclude many individually simple func-tional modules which access and mod-ify a database of moderate complexity inresponse to user commands. The pro-gram code that comprises thecomputer-aided engineering systemthus contains two principal compo-nents: a command decoder and displayunit for communicating with the user,and a collection of functional modulesto carry out the individual data proces-sing functions. The latter is usuallyreferred to as a software package orsimply as a package.

This paper concentrates on the pro-cess of ensuring that the specification ofa software package will be internallyconsistent and will meet its task require-ments. Command interpretation andwork flow control, on the other hand, isnot considered here.

A software package is a collection offunctions needed for a particular appli-cation, such as computer graphics orfinite-element analysis. Packages whichmay appear quite straightforward whenexamined at the component level caneasily become unmanageably complexwhen many components are assembledtogether. Specification of packagecharacteristics therefore presents twodistinct tasks: validation and verifi-cation. Validation, which answers thequestion 'Has the problem been posedproperly?', is not to be confused withverification, which seeks to answer thequestion 'Has the problem been

202

answered properly?'The work described in this paper is

essentially empirical in character and isillustrated by application of its meth-odology to a particular package, theGKS (Graphics Kernel System) graphicsstandard (Ref. 1). A preliminary outlineof the test system described here waspresented earlier at the Third Interna-tional Workshop on Software Specifi-cation and Design (Ref. 2).

Specifications and requirementsengineering

Software engineering is used todescribe the application of scientificmethodologies to the construction ofcomputer software. Requirementsengineering is one aspect of softwareengineering. It endeavours to create aprecise description of whythe system isto be developed, and what it must beable to do. Until recently, engineeringanalysts tended to place emphasis onthe process of creating the programcode that embodies the required func-tions. As packages have grown moreflexible and less structured, however,most workers in the field have tended toview specification as a separate task dis-tinct from software creation, indeed asits necessary pre-condition.

A specification is a document thatdescribes what performance isexpected of the system to . bedeveloped. Many authors (Refs. 3-7)have described the desirable qualitiesof a specification. Nearly all agree that aspecification should be as general aspossible to permit the implementor themaximum freedom. Different imple-mentations of the same specification

should all produce results that satisfythe stated requirements. A specificationmay thus be regarded as an authoritat-ive description of the outputs expectedfrom the system given certain inputstates.

Floyd (Ref. 8), Hoare (Ref. 9), Parnas(Ref. 10) and Dijkstra (Ref. 11) have allwritten critiques or offered authoritat-ive views on verification. Ross (Ref. 12)and Jackson (Ref. 13) have createdmethodologies to aid in the structureddevelopment of software. However,their techniques are tied directly tothe development of program code.Specialised languages have also beendeveloped (Refs. 14 and 15) to facilitatethe description of a specification, buthave not gained wide acceptancebecause they are ill suited to personnelunfamiliar with the mathematical theoryof computer programs or predicate cal-culus. Gerhart and Yelowitz (Ref. 5)point out the inadequacies of several(both specific and generic) program-ming methodologies. Hamilton andZeldin (Ref. 16) express pessimism, sug-gesting that the ultimate evaluation ofsuccess will be made by users withoutdirect reference to stated require-ments.

The specification of software pack-ages is viewed here in terms of oper-ation descriptions and data elements.The validation process is one of com-paring system requirements with aspecification description. But how canrequirements be described adequatelyin a medium understandable to both acomputer professional and a layman?Clarity of communication is of theessence, for if the computer profes-sional cannot understand the systemspecification, or if the description isgiven two different interpretations,then the package user will not likely besatisfied in the end.

Many authors suggest that programsbe dealt with as mathematical objects(Refs. 11 and 17). This implies that forproper design, implementation andmaintenance of software systems anextensive background in mathematicswould be useful, or even necessary.Why does this continue not to be thecase? There has been a certain apparentreluctance in industry to adopt formal

Computer-Aided Engineering Journal October 1986

methods. Within the past decade thisreluctance has certainly decreased, par-ticularly in the software houses, wherethe practical importance of formalmethodologies has grown steadily.Mathematical complexity must be hid-den from users of the methodology toeliminate an explicit need for the under-standing of specification mathematics.This should present a reasonable com-promise, lacking in some commerciallyavailable systems, between mathemat-ical formality and descriptive infor-mality (Refs. 18 and 19).

Parnas and Clements (Ref. 20) pro-posed a number of reasons why thesoftware design process will alwaysremain an idealisation, including thefollowing:

• End-users do not know what theywant.• Many system details become evi-dent only during implementation.• It is difficult to determine all rele-vant facts and partition them so thatthey become easily manipulable.• Human error is difficult to avoidwhen humans are involved.

A difficulty in any specification processis the determination of an adequate,complete and consistent description ofthe desired result. Adequacy is toooften determined post facto, when it istoo late. Many projects are delayed orexperience cost overruns becausemajor specification changes are madeduring implementation. Implemen-tation costs and the degrees of non-sat-isfaction on the part of software usershave been studied by Frank (Ref. 21).Consistency checks ensure that.con-flicting goals are recognised andrationalised.

Levels of specification

The information presented in a specifi-cation will vary from abstract ideas todetailed procedures, and can be classi-fied into a series of levels of abstraction.Abstraction may be considered in termsof the nature of the information com-municated, the environment in whichthe communication is accomplished,and the amount of information com-municated. The specification of pack-ages is often more complex than thespecification of complete end-user sys-tems, because the highest levels ofabstraction are rarely available to thespecifier. One possible hierarchicalorganisation, suitable for engineeringor scientific tasks, is the following set oflevels:

• A stated need for a system that willmaintain and process information: A'need' will define the 'what' of the sys-

^ ^ p « f # T ^

Fig. 1 General overview of the specification structure

tern, but will probably give no indi-cation of what functionality or structureis required. Specification at this level israrely applicable to packages.• A description of detailed require-ments, in terms of the information pro-cessed and the functionality necessaryfor the support of the proposed system:Atthis level a description, invariably in anatural language such as English, willgive details of the functions and struc-tures involved. The official specificationof GKS is stated at this level.• A detailed description of thealgorithms and structures required toprocess and represent the detailedrequirements: This level of specifi-cation provides design information onhow to accomplish previously estab-lished requirements.• The software code to implement therequirements previously stated: Soft-ware code is a description of the pro-cedure actually used. Some develop-ment systems produce implementationcode directly from higher levels ofspecification.

Consistency and completeness check-ing is logically best done at the secondlevel, where the data, functions andrestrictions are known but not yetobscured by detailed procedural needs.The work described in this paper con-centrates on this level. It is based on thepremise that a methodology can bedeveloped and applied which will resultin specifications to meet requirements.

Comparison of real needs with thosespecified is difficult with formalmethods, for there is no formal way ofdefining user needs. Naur (Ref. 22)touches on this subject when he sug-gests that intuition has a valid role toplay in software development. The ulti-mate test of validation is to determinewhether the system specified is accept-able. One approach would be to dem-onstrate the finished product to end-

Computer-Aided Engineering Journal October 1986

users, who would then determinewhether it was satisfactory. Full imple-mentation for trial purposes beingunacceptably costly, a reasonable com-promise is to produce a scenario-basedprototype.

Approach and objectives

While the past decade has brought anincrease in the development of soft-ware design methodologies, most ofthese tools have focused on thedevelopment of software code by someform of synthesis from user require-ments. Nevertheless, prototyping iswidely recognised as a useful tool formanipulation and examination of speci-fications without regard to their even-tual implementation. Prototypes are ofthree main types:

• A prototype is a functional modelused to determine requirements for aproposed system, at a complexity andcost level perhaps one-tenth that of theactual system. Such a prototype is dis-carded and replaced with the oper-ational system. Alternatively, theprototype may also be developed andrefined into the final product directly(Ref. 23).• A rapid prototypes a prototype pro-duced quickly. Klausner and Konchan(Ref. 24) suggest that such a prototypemay not meet all real-time require-ments; however, it should exhibit the'functional behaviour of the target'.• A scenario-based prototype (Ref. 25)simulates events that would be experi-enced by a user of the specified system.Unlike true prototypes, which performlimited useful functions, scenarios ingeneral permit a simulation of thesequence of specified events and input/output operations without actual pro-duction of real results.

The approach taken here is to create a

203

structure into which a software packagespecification may be placed for evalu-ation, both statically and dynamically.Such a structure may be regarded as aprogramming environment particularlywell adapted to the building of sce-nario-based prototypes. It is proposedthat 'templates' be used to filter thespecification information passed fromthe end-user to the structure to simplifythe communication process. A 'tem-plate' is a question posed to the end-user with an appropriate slot for a reply.The following assumptions are made:

• The specification is written in

English or another natural language.• Function descriptions are given interms of associated parameter lists(consisting of input or output par-ameters), a text description of function'effects', and restrictions on inputand/or various 'error' conditions.• Checks are limited to the functionalaspects of the system specification.• Interpretation on the part of theindividual processing the specificationshould not be dependent on a trainingin the field of formal methods.

Under these assumptions, scenario-based prototypes can be constructed

!• ' »•*'. • -"' v '\ *•? ^

* * s ^iJ *Si '*'*"

Fig. 2 The constituent components of a function

204

readily because each data item to bedealt with is described and its relation toevery function is known; conversely,every function is described in relationto the data it accesses or modifies. Theprototypes envisaged are thus true pro-totypes, in that they verify the avail-ability and validity of every data itemneeded for the execution of the system.However, the prototypes are also sce-narios, in that the authors' validationmethod provides the user with a com-mand language through whichsequences of operations can be simul-ated. In other words static validationchecks are included in function simu-lations; dynamic checking is possibleand a 'feel' for the system can beacquired by the playing through ofscenarios.

Proposed methodology: staticchecks

In the following it is assumed that thepackage structure allows it to be dividedinto two parts: information structuresand associated transformations. Infor-mation is embedded in data elements,while transformations are described byfunctions. Each data element isassigned a data type. This attributedetermines the form of data storage inany future implementation, as well asthe operations that may be performedon the data element. Fig. 1 illustratesthe related structure of data elementsand functions.

Functions and data elements areentities of equal importance to thespecification process. Each functionmust have a unique name for reference.Functions are classified by their type,level and state. The function typeattribute classifies functions by theirgeneral nature, for example as controlfunctions or output functions. Thefunction level determines the degree offunction availability for systems thatmay not require a full implementation.The term function state is used todescribe when the function may beapplied to a particular combination ofdata element values. A function appli-cation may be possible in more thanone operational state. The constituentstructure of a function is illustrated inFig. 2.

The notion of data element must notbe confused with variables in programcode. In most cases data elements willcorrespond directly to program vari-ables, but they need not always do so.For example, a Fortran program variablemay be a named array in which variousunrelated data elements are groupedtogether for the programmer's con-venience. Data elements (of the specifi-cation) are pieces of information that

Computer-Aided Engineering Journal October 1986

are internal to the package specified.The nature of the data types requiredfor examination was restricted to thoseneeded for the proposed application ofthe validation process. Each data ele-ment, by its very existence, constitutesa reference that can be checked.

Three descriptors, allocation, defi-nition and value, are used to describehow much is known about each dataelement. All three are Boolean stateindicators. Upon creation of a data ele-ment the state indicators are set to false,so that every data element is initiallyconsidered to be unallocated,undefined and have no value. Whenmanipulating rough ideas, it is easyenough to introduce a new concept andleave it undetailed. Detail is generallyintroduced later in the definition pro-cess. As more information becomesavailable during a scenario generation,at least some state indicators willchange to true.

When a data element is first accessedby a function, it must be initialised. Ifthis was not the case, as an input to afunction it will cause a validation excep-tion to prevail in the validation systemdescribed here. In other words, theoriginal assumption is declared incor-rect and the specification is considerednot consistent, complete and unam-biguous. The result of the output oper-ation will, at the very least, change theallocation state indicator to true to indi-cate that an operation akin to allocationof memory on a computer has takenplace.

If the data element has been defined,then the definition state indicator is setto true. A data element whose stateindicators are true may be used as anargument of a function. If the value of adata element is actually known, thenthe value state indicator is set to trueand the appropriate value is maintainedin the data element. The differencebetween definition and valuation mustbe clearly distinguished. A data elementmay be defined, but its value may beunknown.

All the operations discussed aboveare part of dynamic simulation, not ofthe static definition of data elements.Static definition only includes settingthe data element state indicators to

false. The definition of functions anddata elements establishes a basic struc-ture. A sub-structure is introduced toaccount for a description of the inter-face between functions and dataelements.

Parameters constitute the set of dataelements to be manipulated by a func-tion. They constitute the sole list ofinformation affected by a function(assuming orthogonality of data ele-ments). Parameters are classified inaccordance with the nature of the dataelement referenced. A parameter maybe classified as a function input or afunction output.

It is assumed that no parameter maybe classified as available for both inputand output. This restriction may appearunduly strong at first glance, but is seenon reflection to be less so, for it is theparameter, not the data element, whichis restricted. While a parameter list con-stitutes the set of information that ispassed to or from a function, otherinformation internal to the system mayalso be affected. This information mustbe identified, even when it is internal tothe system and does not appear in theexternal parameter list (however, itmust appear in the parameter list ofsome function to be considered as con-tained in the specification). Thus aparameter is designated as either inter-nal or external, depending on whetheran explicit reference is made to it in thefunction parameter list.

Effects represent the operation that afunction performs, within the limitationthat no description is given of how theoperation is carried out. In other words,implementation concerns are notconsidered.

An effect is considered to describethe processing of a set of data elements,the function parameters. An effectplaces requirements on the allocation,definition and value indicators. Theseindicator values form the basis of check-ing the dynamic interplay of the func-tions and data elements. Violations ofthe value requirements are flagged asvalidation exceptions during thedynamic checking process.

Effects are sub-divided into fourclasses: transformation effect, testeffect, test and transform effect and

effect not classified. This classificationis used to establish the manner in whichthe effect is dynamically simulated. It isassumed, of course, that the functionswhose effects are to be described aredeterministic in nature.

A restriction is an imposed limitationon the value that a data element or aparameter might possess. Mathemat-ically, a restriction is a domain defi-nition. Restrictions are imposed at twodistinct levels: on data elements and onparameters. Numeric and class mem-bership restrictions are provided.

Numeric restrictions impose limits onthe domain of validity of input. Forexample, a graphics device might limitco-ordinates to positive integers lessthan some specified value n, or in therange of values (1.. .n). However, aparameter referencing the same dataelement might require that input be inthe domain ( 1 . . . m), where m is con-tained in the set ( 1 . . . n). Membershipconcerns arise in connection with enu-merated data types. An enumerateddata element must have one of a set ofpossible values, not necessarilyordered. The structure of dependencebetween data elements, functions, par-ameters, restrictions and effects is illus-trated in Fig. 3.

Dynamic checks

Dynamic specification checks areunderstood to operate at several levelsof complexity. It is important to identifyand resolve any possible conflictsbetween functions. Similarly, the end-user must be given an opportunity toexperience the system prior to its imple-mentation. On the other hand, it is notthe objective of the validation scenarioprocess to produce a working pro-totype with all the features andcharacteristics of the specified system.

Dynamic checks and simulations aredone by manipulating the definitions offunctions as given in terms of the stati-cally defined effects. The followingsummarises the desirable checks:

• Verify that the function is callable inthe current operating state of the sys-tem (it is assumed that the operatingstate will be set in the same fashion as in

—-function classification-—•

- S i . * . H

f.7 -f Ka.§lePf n^funttion-

^ ~ level

Fig. 3 Integration of data elements and functions

Computer-Aided Engineering Journal October 1986 205

the implemented system, i.e. by func-tion calls; similarly, checks may bemade dependent on the level ofimplementation).• After presentation of the par-ameters to the user and input acquisi-tion, effect processing begins.Determine that each data elementdescribed is in its correct input state,i.e. that the usage of the Boolean stateindicators (allocation, definition andvalue) is consistent. Inconsistenciesrecognised in the specification (forexample, the specification may indicatethat a special output in the form of anerror message should be issued) andthose determined by checking thespecification are distinguished.

• Natural language text associatedwith each effect is displayed as pro-cessed. The Boolean state indicatorsassociated with data elements are set asprescribed.

Experimental implementation

A specification checking structure ofthe kind described above has been con-structed by the authors and used todetermine whether a proposed specifi-cation is valid in terms of consistency,completeness and lack of ambiguity.The test system has been implementedusing the INGRES database manage-ment system running under UNIX on aDEC VAX 11/750 computer.

This choice of software and hardwareis essentially arbitrary; however, theGKS specification served as the testvehicle for this work. It is appropriatebecause it incorporates a large numberof the features commonly associatedwith engineering and scientific soft-ware, particularly interactive software:modularity, function independence,orientation to cleanly structured data,and implementation at several differentlevels and states. GKS is defined by aprecise document, so there is littledoubt about its specification, whichremoves one source of difficulty inspecification checking. In the interestof flexibility, the main part of the workhas concentrated on a low-level imple-mentation of GKS, so that system testscan be carried out easily and modifica-tions in the scenario generator madereadily, without becoming tangled inexcessively large data structures.

Observations and conclusions

After extensive months of trials, thescenario-based prototype approachdescribed here appears to be satisfac-tory for locating inconsistencies — andperhaps more importantly incon-veniences which are not formallyerroneous — in software package speci-

fications. To conclude, several obser-vations may be stated:

• Consistency checks are useful, not afutile chase of the obvious, even if thesystem specified is only moderatelycomplex. In the case of GKS there are

^approximately 150 functions and 300data elements — the specification ofGKS (Ref. 1) provides lists of functionsordered by state, level etc. as additionalinformation.• The templates used to support thevalidation process were an effectivemeans of enforcing consistency as thespecification was processed.• The activity of processing specifi-cations can rapidly become tedious,

particularly if system response time is afactor. This tedium is difficult to avoid;however, if it is relatively simple tomanipulate the specification, it can beminimised.• The limitations of scenario-basedprototyping should be recognised. Inparticular, the temptation of making thescenario too good must be resisted, lestit begin to approach a true prototypeimplemented in the wrong language.

Acknowledgment

The authors wish to acknowledge thefinancial support of the Natural Sci-ences and Engineering Research Coun-cil of Canada.

References

1 ACM SICCRAPH: 'GKS: Graphical Kernel System', Computer Graphics, 1984, 18, Feb.2 FRASER, S. D., and SILVESTER, P. P.: 'An interactive empirical approach to the validation

of software package specifications'. Proceedings of Third IEEE Computer Society Interna-tional Workshop on Software Specification and Design, 26th-27th Aug. 1985, pp. 60-62

3 BALZER, R., and GOLDMAN, N.: 'Principles of good software specification and theirimplications for specification languages'. Proceedings of IEEE Conference on Specifi-cation of Reliable Software, IEEE Cat. No. 79CH1401-9C, 1979, pp. 58-67

4 BASILI, V. R., and MILLS, H. D.: 'Understandingand documenting programs', lEEETrans-actions on Software Engineering, 1982, SE-8, (3), pp. 270-283

5 GERHART, S., and YELOWITZ, L.: 'Observations of fallibility in applications of modernprogramming methodologies', ibid., 1976, SE-2, (3)

6 LINGER, R. C , MILLS, H. D., and WITT, B. I.: 'Structured programming: theory andpractice' (Addison-Wesley, 1979)

7 PARNAS, D. L.: 'The influence of software structure on reliability', in YEH, R. T. (Ed.):'Current trends in programming methodology. Vol. 1' (Prentice-Hall, 1977), pp. 111-119

8 FLOYD, R.W.: 'Assigning meanings to programs'. Proceedings of American MathematicalSociety Symposium in Applied Mathematics, 5th-6th April 1967, Vol. 19, pp. 19-32

9 HOARE, C. A. R.: 'An axiomatic basis for computer programming', Communications ofACM, 1969, 12, (10), pp. 576-583

10 PARNAS, D. L.: 'A technique for software module specification with examples', ibid.,1972, 15, (5), pp. 330-336

11 DIJKSTRA, E. W.: 'A discipline of programming' (Prentice-Hall, 1976)12 ROSS, D. T.: 'Structured Analysis (SA): a language for communicating ideas', IEEE Trans-

actions on Software Engineering, 1977, SE-3, (1), pp. 16-3413 JACKSON, M.: 'System development' (Prentice-Hall, 1983)14 CLOCKSIN, W. F., and MELLISH, C. S.: 'Programming in Prolog' (Springer-Verlag, 1981)15 GOGUEN, J. A., and TARDO, J. J.: 'An introduction to OBJ'. Proceedings of IEEE Con-

ference on Specification of Reliable Software, IEEE Cat. No. 79CH1401-9C, 1979, pp. 170-189

16 HAMILTON, M., and ZELDIN, S.: 'Higher order software — a methodology for definingsoftware', IEEE Transactions on Software Engineering, 1976, SE-2, (1), pp. 9-32

17 JONES, C. B.: 'Software development: a rigorous approach' (Prentice-Hall, 1980)18 BALZER, R., GOLDMAN, N., and WILE, D.: 'Informality in program specifications', IEEE

Transactions on Software Engineering, 1978, SE-4, (2), pp. 94-10319 GRIES, D.: 'The science of programming' (Springer-Verlag, 1981)20 PARNAS,D. L., and CLEMENTS, P. C : 'A rational design process: how and why tofake it',

in EHRIG, H., FLOYD, C , NIVAT, M., and THATCHER, J. (Eds.): 'Formal methods andsoftware development'. Lecture Notes in Computer Science, Vol. 186 (Springer-Verlag,1985), pp. 80-100

21 FRANK, W. L.: 'Critical issues in software' (John Wiley, 1983)22 NAUR, P.: 'Intuition in software development', in EHRIG, H., FLOYD, C , NIVAT, M., and

THATCHER, J. (Eds.): 'Formal methods and software development'. Lecture Notes inComputer Science, Vol. 186 (Springer-Verlag, 1985), pp. 60-79

23 BLUM, B. I., and HOUGHTON, R. C : 'Rapid prototyping of information managementsystems', SICSOFT Software Engineering Notes, 1982, 7, (5), pp. 35-38

24 KLAUSNER, A., and KONCHAN, T. E.: 'Rapid prototyping and requirements specificationusing PDS', ibid., 1982, 7, (5), pp. 96-105

25 HOOPER, J. W., and HSIA, P.: 'Scenario-based prototyping for requirements identifica-tion', ibid., 1982, 7, (5), pp. 88-93

S. D. Fraser and Prof. P. P. Silvester are with the Computational Analysis & Design Laboratoryof the Department of Electrical Engineering, McGill University, 3480 University Street,Montreal, Quebec, Canada H3A 2A7.

206 Computer-Aided Engineering Journal October 1986