Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | michael-weaver |
View: | 222 times |
Download: | 0 times |
Application Domain Knowledge and Programmers’ Mental Representations
Teresa M. ShaftUniversity of Oklahoma
Iris VesseyIndiana University
Application Domain Knowledge
Application Domain refers to the context of the problem to be addressed by the computer software – in contrast to the solution or computing domain (Blum 1989)
adapted from Blum, 1989
Implementation
Domain
Application
Domain
ConceptualModels
FormalModels
The Software Development Process
Importance of Application Domain Knowledge in Software Development
Those who can map deep application domain knowledge into a computation solution often serve as project gurus
“the thin spread of application domain knowledge” was one of the most salient problems in large scale development Curtis, Krasner & Iscoe 1988
Significant effort during software development is associated with acquiring needed application domain knowledge Walz, Elam & Curtis 1993
Application Domain Knowledge Acknowledged as critical to the ability to
develop, comprehend, and maintain software (Brooks, 1990, Pennington & Grabowski 1990, Vans, von Mayrhauser & Somlo 1999)
Infrequently studied (Glass & Vessey 1998).
Most studies of software comprehension or maintenance either do not consider the application domain or use materials the researchers argue do not require application domain knowledge.
Application Domain Knowledge in Software Comprehension
Programmers use a more top-down (hypothesis driven) comprehension process when they have relevant application domain knowledge (Shaft & Vessey 1995)
During software comprehension, programmers use application domain knowledge in addition to (rather than to replace) programming knowledge (Shaft & Vessey 1998)
Current Study
Examines the influence of application domain knowledge on programmers’ mental representations during the software comprehension and enhancement process
Examines the influence of different types of enhancement tasks on the development of programmers’ mental representations
Comprehension Process
Program Code and
Documentation
Mental Representation
of Computer Program
Enhancement Specification
Enhancement Process
Mental Representation
of Enhanced Program
Enhanced Computer Program
Application Domain
Knowledge
Programming Domain
Knowledge
Programmer’s Knowledge Base
Program/ProgrammerSpecific Knowledge Base
Program/ProgrammerSpecific Knowledge Base
Conceptual Model of the Software Comprehension and Enhancement
Programmers’ Mental Representations
Internal knowledge structure of the contents of the computer program
Numerous types of information are embedded in a computer program (Brooks 1987, Pennington 1987, Green 1977, Corritore & Wiedenbeck 1999)
Comprehension is the process of extracting that information from the computer program
Programmers’ mental representations reflect their comprehension of the different types of information embedded in the programs
Knowledge Categories
Function Data flow Control flow State
Pennington 1987a, 1987b These categories are relevant to
procedural programs
Initial Comprehension of a Computer Program
Driven by the knowledge programmers’ bring to the comprehension task
Hypothesis 1: Initial Comprehension H1a: Programmers have higher levels of
initial comprehension when they are familiar with the application domain of the program than when they are unfamiliar with the application domain.
H1b: When programmers are familiar with the application domain, they have better comprehension of function and data flow information than of control flow and state information.
Comprehension After Conducting an Enhancement
Influenced by the nature of the enhancement task as well as the knowledge programmers’ possess
Different types of enhancement tasks rarely considered
Consider two types of enhancement tasks: Function – high need for application
domain knowledge Control flow – low need for application
domain knowledge
Hypotheses 2: Comprehension After Conducting An Enhancement H2a: Comprehension after conducting an
enhancement will be greater when programmers are familiar with the application domain (than when they are unfamiliar with the application domain).
H2b: Programmers have greater comprehension of the type of knowledge emphasized in the enhancement task than the other types of knowledge. Control flow task -> control flow knowledge Function task -> function knowledge
Changes in Programmers’ Comprehension
Influenced by programmers’ knowledge
Influenced by the nature of the enhancement task
Hypotheses 3: Changes in Comprehension
H3a: Programmers experience greater changes in comprehension when they are familiar with the application domain (than when they are unfamiliar)
H3b: Programmers who conduct a control flow task have greater increases in comprehension than those who conduct a function task
Methodology 24 professionals studied and
enhanced two computer programs Minimum of 2 years professional
experience Computer programs from two
application domains Familiar application domain: accounting Unfamiliar application domain: hydrology COBOL programs Equivalent size, data density, decision
density
Methodology (continued) Assessed programmers’
comprehension at two points: After an initial study period After conducting an enhancement task
Comprehension assessed via questions Two questionnaires per program 20 questions per questionnaire 5 questions for each knowledge
category
Methodology (continued)
Half of the programmers conducted control flow enhancements on both programs; half conducted function enhancements
Control flow tasks: insert a new level of control break in an existing control break report
Function tasks: create a new capability Enhanced programs of equivalent size,
data density & decision density
Analysis of Hypothesis 1
Source DF SS Mean Square F Value Pr > F
Application Domain (AD) 1 14700.00 14700.00 16.74 .001Error 23 20200.00 878.26
KnowledgeCategory (KC) 3 8625.00 2875.00 7.61 .01Error 69 26075.00 377.90
AD * KC 3 466.67 155.56 .44 .73Error 69 24633.33 357.00
Hypothesis 1 – Initial Comprehension H1a: supported
programmers’ have higher levels of comprehension in the familiar application domain
H1b: not supported No interaction between knowledge category
and application domain familiarity Programmers’ comprehension of state
knowledge was significantly greater than data flow, regardless of knowledge of the application domain
Initial Comprehension by Knowledge Category
70.84
59.17
69.1773.33
53.33
38.33
50.00
30
40
50
60
70
80
90
100
Function Data Flow Control Flow State
Knowledge Category
Percent CorrectFamiliar(Accounting)Unfamiliar(Hydrology)
Analysis of Hypothesis 2Source DF SS Mean Square F Value Pr > F
Type of EnhancementTask (TET) 1 1752.08 1752.08 1.44 .24Error 22 26845.83 1220.27
Application Domain (AD) 1 10502.08 10502.08 31.21 <.001AD * TET 1 4680.75 468.75 1.40 .25Error 22 7389.58 335.42
KnowledgeCategory (KC) 3 7289.58 2429.86 10.81 <.001KC * TET 3 522.92 174.31 .78 .51Error 66 14837.50 224.81
AD * KC 3 5972.92 1990.97 6.65 .001AD * KC * TET 3 2106.25 702.08 2.34 .08Error 66 19770.83 299.56
Hypothesis 2 – Comprehension After Conducting the Enhancement
H2a: supported Comprehension in the familiar application domain
is greater than in the unfamiliar application domain H2b: not supported
Type of enhancement task did not influence which knowledge categories are better comprehended
Application domain knowledge influenced comprehension of the different knowledge categories: Data flow, control flow and state knowledge more
accurately understood in the familiar application domain
Within familiar application domain: state knowledge more accurately understood than other categories
Within unfamiliar application domain: state and function more accurately understood than data flow knowledge
Application Domain Familiarity by Knowledge Category After Enhancement Task
86.67
72.50 71.67
66.67 65.00
57.50
45.83
70.00
30
40
50
60
70
80
90
100
Function Data Flow Control Flow State
Knowledge Category
Percent Correct Familiar (Accounting)
Unfamiliar (Hydrology)
Application Domain:
Analysis of Hypothesis 3Source DF SS Mean Square F Value Pr > FType of Enhancement Task (TET) 1 176.04 176.04 0.06 .81Phase 1 5551.04 5551.04 16.14 <.001TET * Phase 1 2109.38 2109.38 6.13 .02Error 22 7564.58 343.84Application Domain (AD) 1 25026.04 25026.04 38.85 <.001AD * TET 1 26.04 26.04 0.04 .84Error 22 14172.92 644.22 Knowledge Category (KC) 3 15586.46 5195.49 15.51 <.001KC * Task 3 786.46 262.15 0.78 .51Error 66 22102.08 334.88Phase * AD 1 176.04 176.04 0.29 .59Phase*AD*TET 1 651.04 651.04 1.09 .31Error 22 13197.92 599.91 Phase*KC 3 328.13 109.38 .41 .75Phase*KC*TET 3 969.79 323.26 1.21 .31Error 66 17577.08 266.32ADK * KC 3 3253.13 1084.38 4.05 .01ADK* KC* TET 3 1553.13 517.71 1.93 .13Error 66 17668.75 267.31Phase*ADK*KC 3 3186.46 1062.15 2.65 .06Phase*ADK*KC*TET 3 844.79 281.60 .71 .55Error 66 26443.75 400.66
Hypothesis 3 – Changes in Comprehension H3a: not supported, application domain
knowledge did not allow programmers to gain more knowledge
H3b: supported, those who conducted a control flow enhancement showed greater gains in comprehension
Application domain influenced the types of knowledge programmers understood: Higher levels of comprehension of all knowledge
categories in the familiar application domain Within familiar application domain: state
knowledge more accurately understood than function and data flow
Within unfamiliar application domain: function, control flow and state knowledge more accurately understood than data flow
Application Domain Familiarity by Knowledge Category Interaction Across Phases
80
68.75
65.42
70.83
61.67
42.08
53.75
62.92
30
40
50
60
70
80
90
Function Data Flow Control Flow State
Knowledge Category
Percent Correct Familiar (Accounting)
Unfamiliar (Hydrology)
Application Domain:
Phase by Type of Enhancement Task Interaction
70 Control Flow
57.71
63.96 Function 61.04
30
40
50
60
70
80
Prior to Enhancement After Enhancement Task
Phase
Percent Correct Control Flow Function
Type of Enhancement Task:
Summary of ResultsResearch Question 1: Initial
Comprehension2: Comprehension After
Conducting the Enhancement 3: Changes in Comprehension
Application Domain
Familiarity
Familiar > unfamiliar
Familiar > unfamiliar. Application domain knowledge did not result in greater gains in
comprehension. Type of
KnowledgeState > data
flow Across Application Domains:
Data flow, control flow & state knowledge better understood in the familiar than unfamiliar
application domain.Within Familiar Application
Domain: state > function, data flow & control flow.
Within Unfamiliar Application Domain: function & state >
data flow.
Across Applications Domains: all knowledge categories better
understood in the familiar application domain.
Within Familiar Application Domain: state > function & data
flow. Within Unfamiliar Application Domain: function, control flow
& state > data flow.
Type of Enhancement
Task
Not applicable Type of enhancement task did not influence programmers’
comprehension or their understanding of different
types of knowledge
Conducting a control flow enhancement increased overall
comprehension.
Discussion
When programmers possess application domain knowledge they have higher levels of comprehension initially and after enhancing a program Consistent for two types of enhancements
Even in the context of program maintenance, a software development task that is seen as relatively far from the original domain, relevant application domain knowledge benefits software comprehension
Discussion
Conducting a control flow enhancement task led to gains in comprehension Not all types of tasks will give
programmers’ the same opportunity to develop their mental representation of a computer program
A more complete taxonomy of tasks would be helpful to future researchers
Implications – Types of Enhancement Tasks
Long-term vs. short-term employees Programmers’ mental representations
develop if they engage in tasks that require changes to multiple locations of the program
Programmers who conduct enhancements which do not require interacting with much of the original program, did not experience significant increases in comprehension
Implications – Application Domain Knowledge
Lengthy start-up time for new IT employees
Training and selection Outsourcing and off-shoring
Activities that require less application domain knowledge would seem to be better choices
Develop mechanisms to provide application domain knowledge
Conclusions Not all enhancement tasks encourage
development of programmers’ mental representation of a computer program.
Traditionally, software comprehension and enhancement were seen as residing in the programming domain and requiring little, if any, application domain knowledge.
Application domain knowledge is beneficial to software comprehension, and to the understanding of all knowledge categories.