Date post: | 03-Jun-2018 |
Category: |
Documents |
Upload: | govind-kamboj |
View: | 226 times |
Download: | 0 times |
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 1/59
Software Reviews,
Walkthroughs, Inspections, and
AuditsBetter to find an error twice than
never to find it at all.
[Freedman/Weinberg 90] Freedman,
D.P., G.M. Weinberg, "Handbook of
Update terminology
according
Give references
to O’Neil
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 2/59
Why Do We Have Formal
Technical Reviews?
1. To err is human.2. Lots of errors escape the originator more
easily than anyone else.
3. Reviews are educational.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 3/59
Purpose: Provide
• visibility into state of project
• opportunities for personnel (project and
non-project) to discuss topics related to
project
• intermediate milestones and sense of
progress
• assessment of technical adequacy of project
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 4/59
Use the power of a team to …
• Point out needed improvements of a
product.
• Confirm the parts in which improvement is
not needed or desired.
• Make technical work more uniform and
predictable in quality, which makes it moremanageable.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 5/59
Technical review is different
from other reviews• For example, budget and management
reviews
• Technical review answers question: will this
product do the job it’s supposed to do
• If answer is ―no‖, then
– no schedule is on time, and
– no cost is cheap enough.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 6/59
When?
• Single review that occurs on a particular date
• Single review that occurs in response to a
particular condition• Multiple reviews that occur periodically: e.g.,
monthly
• Multiple reviews that occur in response to a particular condition, e.g., code review forsubsystem
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 7/59
Benefits
• 10X reduction in errors in products
• 50-80% cost reduction
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 9/59
Why not just pass it around and
have reviewers sign off …
• Comments are too large• Comments are unread
• Reviewers don’t have to take responsibility
for the content.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 10/59
Requirements
• Team participation
• Documented procedures
• Documented results
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 11/59
Types of Reviews
• Management Review
• Technical Review
• Walkthrough
• Inspection
• Audit
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 14/59
Types of Reviews
• Management Review
• Technical Review
– Focus on specification and design
– Determine whether products meet specs
• Walkthrough
• Inspection
• Audit
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 15/59
Scenario for a typical review
• Leader: Welcome. We are here to review X for the
purpose of Y.
• Each person is introduced.• The recorder is introduced.
• An agenda is presented and the review process
explained.
• The list of material in the review packet is stated.
• The review begins…
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 16/59
Review
• Agenda set by leader , agreed to by
committee
• Review proceeds line-by-line.
• Reviewers bring up issues as they encounter
them.
• Issues are added to an Issues List.
• Agenda changes as issues are encountered.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 17/59
Round Robin Review
• Force every member to participate.
• Each person takes the lead for a section:
– paragraph,
– line of code,
– function,
– test case
• Good way to get involvement from team.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 18/59
Types of Reviews
• Management Review
• Technical Review
• Walkthrough – Used to find anomalies and evaluate conformance
– Typically used for source code
– Effective form of education
• Inspection
• Audit
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 19/59
Walkthroughs
• Producer guides review
• Step by step presentation of product:
– Code, design, report, test cases … • Can cover lots of material
• Can have more people attend, less prepared
• More work for presenter
• May be difficult to control interactions
– Too many interruptions
– Always hard to have a producer present
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 20/59
Walkthroughs
• Participant should spend 1 hour preparing
for each hour in meeting
• Should be scheduled to be brief
• Should only review completed code or
document
• Should require participants to sign report
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 21/59
Types of Reviews
• Management Review
• Technical Review
• Walkthrough
• Inspection
– More formal type of walkthrough
– Used to identify specific errors or problems
• Audit
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 23/59
Types of Reviews
• Management Review
• Technical Review
• Walkthrough
• Inspection
• Audit
– Independent evaluation of process or product
– Ensures that the process is being followed
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 24/59
Review Teams
• Permanent or semi-permanent teams
• IV&V or QA teams
• Get to be very good at reviews
• Need to have some way of reviewing the
reviewers.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 25/59
The team• Leader
– Job: obtain a good review or report why it wasnot possible
• Reflects the quality of the review process,not the quality of the product
• Must be accurate
• Recorder
– Provide info for accurate report of review
– Short, public notes – Capture essence of each issue
– Must ensure group has reached conclusion
– Don’t video tape
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 26/59
Tasks of leader
• Monitor preparedness of team members
• Set pace
• Keep meeting on track• Poll members to reach consensus
• Ensure participation
• Has right to terminate review if it is unproductive – Disagreements
– Fatal flaws
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 27/59
Consensus
• Decision of the group is equal to the most
severe opinion of the group members.
• Be conservative.
• Accept the doubts of the most doubting
member.
• No ―I told you so, but I got outvoted‖.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 28/59
Reviewers
• Must be qualified to contribute
• Must be unbiased
• Don’t invite management if it causes
conflict
– The point is to review the project, not the
producers.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 29/59
Rules for Reviewers
I. Be prepared
II. Evaluate product, not people.
III. Watch your language.
IV. One positive, one negative.
V. Raise issues, don’t resolve them.
VI. Record all issues in public.
VII. Stick to technical issues.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 30/59
Rules for Reviewers
• If you find something, assume it’s a mistake.These are your peers, not your enemy. Remember
people are involved
• Avoid ―why did you …‖ why didn’t you …‖ say
instead ―I don’t understand …‖ – ―Why did you set the upper bound to 10 here?‖
– ―I don’t understand why the upper bound is 10 here.‖
– seems trivial, but it’s not.
• Do not get bogged down in style issues. Forexample, if efficiency isn’t an issue, then don’tmake it one.
• If it makes things less maintainable, that’s an issue.
• If there are standards, then either stick to thestandard, or dispose of the standard.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 32/59
Time
• At most 2 hours at a time
• Come back again if necessary
• Depends on
– Complexity
– Size of project
– Closeness to completion
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 33/59
Everyone must prepare!
• Review packet
• Everything relevant to making correct
judgment: code, specs, test data, results,standards, designs, …
• 80% of failures stem from unprepared teams
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 34/59
Tactics for participation
• Devil’s Advocate
– One person makes strongest case against a product
– Job is to find fault
– Needs to be an actor• Debugging
– Producers leave known bugs in
– Quality of review depends on how many found
• Alarms
– Time each person’s contribution, cut off
• Stand-up Reviews
– Talk as long as you stand on one leg
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 35/59
Report
• Goal: Does the product do the job its supposed todo?
– Complete.
– Correct.
– Dependable as basis for other work
– Measurable for purposes of tracking.
• What was reviewed?• Signatures of reviewers.
• Leader and recorder.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 36/59
Notes
• Requires learning: to review and to write
• Need to allocate time
• Not a form of project management
– But can provide information for tracking
• Review early and often
– But not too often
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 37/59
Need for Reviews
• Reviews help organization deliver quality product
– Examine individual parts early in process – Identify problems spanning multiple items or
phases
• Reviews help identify problems early
• Reviews help build image of the productand the vision of the product in the minds ofthe team members
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 38/59
Cost of Software Reviews
• Up-front cost for – Training
– Staff preparation
– Review Conduct
• Code review may be 8-15% of total cost of project
• Offset by savings: reduced rework later in project(25-35% of total project cost saved) – Raytheon: pre-1988: 43% of software cost in defect
correction – 1994 (after software reviews instituted): 5% of project
cost in defect correction
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 39/59
Writing Issues
• Groups of 3
• I’ll give you an issue from an issue list,
• You tell me what’s wrong with it.
• Issues (and responses) from Handbook of
Walkthroughs, Inspections, and Technical
Reviews, Daniel Freedman and Gerald
Weinberg, 3rd Ed, Dorset House.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 40/59
Writing Issues #1
Some of the explanations of user commands
were misinterpreted by members of the
review committee.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 41/59
Comments on Issues #1
Not specific enough. Which explanations were
misinterpreted? Don’t make the producers
guess, or they may change the wrong thing.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 43/59
Comments on Issues #2
In writing, clarity isn’t the most important
thing: it’s the only thing. If you don’t want
to be not understood, don’t never use nodouble negatives.
Another thing – what is the issue anyway? Is
the maximum too low? Too high? Or is itthat some of the codes should have been
made not eligible? Or eligible? Be direct.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 44/59
Writing Issues #3
The referenced table of constant values was
not part of the review packet.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 45/59
Comments on Issues #3
Always give the most direct reference
available. There could be more than one
table, now or in the future. Why make the producers search?
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 46/59
Writing Issues #4
The method used for maintaining the message
queue seems to solve a severe performance problem we’ve been having with the
production version of the RKY system.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 47/59
Comments on Issues #4
This is an interesting observation, clearly and
directly stated. But what does it have to do
with the product under review? It might beworth millions, but it doesn’t belong on the
Issues List for this product. It should go on
the Related issues list.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 48/59
More issues #5The price/performance table should be sorted
using either Quicksort or Shell sort.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 49/59
Comments #5
Raise issues, but avoid all temptation to giveadvice in the Issues List. They won’t be
welcome if they come in that form, so ifyou really must give advice, find someunofficial vehicle. Take the producers tolunch, or out for a beer, before you share
your vast experience on matters of sorting.If your idea isn’t worth the price of a beer,why not forget it?
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 50/59
More issues #6
The three diagrams drawn by Harold Mitter
are not in the standard format (DS-109)
required for such diagrams in our
installation.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 51/59
Comments #6
This point is nicely specific, but why do we
have to mention poor Harold? We are
reviewing the product, not the people. Findanother way to identify the diagrams and
leave people’s names out of the Issues List.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 52/59
More issues #7
The committee was unable to understand the
significance of paragraph 3.1.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 53/59
Comments #7
Nothing’s wrong with this one. It’s specific aboutwhich paragraph is under discussion, and it saysthe committee doesn’t understand it. Who can
argue with that? Yet, surprisingly, people seem to be afraid to express issues this way – ―We don’tunderstand …‖ Don’t worry about being thoughtstupid.
If you don’t understand it, it’s at least a potentialissue in documentation. And besides, lack ofunderstanding may mean there is somethingdreadfully wrong.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 54/59
Yet more issues #8
The bubble sort used for sorting the table of
price/performance figures is a stupid
approach if the table should grow any
bigger than the present 20 entries.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 55/59
Comments #8
The word ―stupid‖ doesn’t add anything at all and
might antagonize the producers. Take it out. Then
try to express factually and quantitatively why the
method is inappropriate for larger tables.
And if the approach is stupid, what of it? As Arthur
C. Clarke expressed it, ―It has yet to be proved
that intelligence has any survival value.‖
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 56/59
Yet more issues #9
If the BY NAME option is not used, the
structuring of the structure operands must
be equivalent to the structuring of the
structures in the arrays of structures.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 57/59
Comments #9
This wasn’t really taken from an Issues List,
but from an old PL/I manual. Still, I’ve read
real issues that were almost as obscure. I’dadvise you to follow the KISS axiom
(―Keep It Simple, Stupid), except that I
would be contradicting my own advice fromthe previous question.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 58/59
Yet more issues #10
Frieda Sonntag has no computer science
background, and her experience with PL/Iis nil. She should never have been
assigned the coding of this module.
8/12/2019 Lecture 16a Walkthroughs
http://slidepdf.com/reader/full/lecture-16a-walkthroughs 59/59
Comments #10
It’s non of the business of the review
committee who management has assigned
to particular jobs. The committee’s businessis to review the product and tell what state
it’s in. How it got to that state is another
issue — and not for the Issues List. Who isresponsible is even less of an issue, and
raising it is sure to be ineffective.