Post on 29-Nov-2014
description
transcript
BT2 Concurrent Session 11/14/2013 10:15 AM
"Contextually-Driven System Architecture Reviews "
Presented by:
F. Michael Dedolph Levi Deal Consulting
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Michael Dedolph Levi Deal Consulting
Michael Dedolph is self-employed as a consultant and innkeeper. For the past seven years, Michael led quality process improvement efforts at CSC and Lucent. Previously, he was a systems architecture review leader at Bell Labs (Lucent) and facilitated numerous risk identification and project retrospective sessions. Prior to 1997 Michael worked in the Risk and Process programs at the SEI where he was the technical lead for teams that developed the SCE and CBA-IPI appraisal methods. His IT career began with ten years as an Air Force computer officer, working in satellite systems, management information systems, and as an instructor of software engineering at the Air Force Institute of Technology.
1 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 1
Contextually-Driven
Architecture Reviews
For Better Software Conference East
F. Michael Dedolph fmdedolph@netscape.net
14 November 2013
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 2
Author’s Note
Most of this presentation material was used previously in these venues.
– SEI’s SATURN 2009 conference – Washington DC Area SPIN meeting in Feb 2009 – PNSQC 2010
Architecture reviews were a good idea then, and they still are
2 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 3
What is “Architecture”? Ø Before we can review it,
we should know what “it” is
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 4
“A System Architecture . . .” Provides a solution to a problem for a client – Early on, it is a conceptual or potential solution
Architecture has also been called “design with constraints” – Alternatively, you could say architecture includes
a design that works within the constraints, including cost and schedule!
Ø Any given architecture is NOT the ONLY solution, but, some solutions are better than others
3 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 5
Where Architecture Exists • Architecture exists at the intersection
of management, technology, and design
Management Technology
Design
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 6
Architecture Problem Statements • The problem space has multiple
aspects: “FFETO” – Function – what it does, how well it does it. – Form – what it “looks like”; includes major
environmental interfaces – Economy – cost, including development,
maintenance, material, system retirement, etc. – Time – relationship to past, present, and future – Operational/ Developmental – Things that
pertain to development constraints and system operating environment, rather than the system
4 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 7
Problem Statements Ø are the basis for the review! Provide a succinct summary of critical
success criteria – Short: 2 pages is good – Includes major constraints – Client-centric – Cover Function, Form, Economy, Time, and
(optionally) Operational – Be expressed in sufficient detail to make
judgments about the proposed solution, but are not unnecessarily proscriptive of prescriptive
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 8
Problem Statements Ø Are NOT – A requirements document, but may summarize
the most critical high level requirements
Problem statements also have unstated "always" criteria, e.g.:
– Possible to construct – Can make money/ will provide value – Solution won't result in harm – Legal/ regulatory constraints
Ø The team must understand these as well.
5 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 9
Prob. Statements Provide Context • Problem statements can be applied to
almost any product or service. • Advantage: representational
independence • Many architecture review techniques depend on the way
the design is represented. • But, design can be represented in many different ways
—e.g., network diagrams, UML, physical prototypes, circuit diagrams, flow charts, use cases . . .
Ø No representation captures all aspects of the system, and focusing too much on representation can actually obscure problem areas
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 10
Classical Architecture -
6 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 11
Taj Problem Statement Included: Function: tomb, mosque, hotel, memorial Form: symmetry, untarnished finish,
unprecedented weight, shearing force of the river, visual perspective maintained
Economy: irrelevant, it will cost what it costs Time: it will take as long as it takes, but must
last for all time Operational: expertise is needed from other
kingdoms, supply chain issues
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 12
Architecture Review Method, in a Nutshell • Define the problem the client wants
solved • Compare it to the architecture
(proposed solution) • Identify the gaps (or risks) Then: Ø Let the project resolve the gaps
7 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 13
Architecture Review Questions Questions to answer before a review: • Who decides? (Know the client) • What problem are we trying to solve? • Are key stakeholder interests represented?
Questions to answer during a review: • How good is the proposed solution? • What are the issues/ risks/ gaps in the solution
Questions to answer after a review: • Which things will we (the project team) address? • How will we (the project team) address them?
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 14
Ground Rules for Reviews Ground rules are covered in the pre-review, and again in the review meeting • No attribution—review products, processes, and
ideas, not people • Team is there to identify, not solve problems • Client requests (and pays) for the review, but,
the project owns the findings and responsibility for correcting them Ø One exception, “Management Alert” (covered later)
• Reviewers can ask questions at any time
8 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 15
Review Steps • Preparation
• Respond to initial contact/request; develop problem statement; select team and develop agenda; arrange logistics – travel, space, connectivity; pre-review call
• Review Meeting • Project presentations with Q&A; review team Caucus;
conduct readout; client meeting (optional)
• Follow-up • Write and distribute the Review Report; present
findings to lessons learned/governing groups (Board Report); review the Project’s action plans; Close the review by meeting with project and/or sponsor
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 16
Preparation – Some Thoughts Toughest part: Problem Statement • I have seen projects cancel a review because
they “didn’t have time to develop a problem statement” How do you think the project fared?
• In many cases, the initial review schedule was delayed while the project and client worked out the problem statement
• The Problem Statement is reviewed in detail during the pre-review meeting, used to finalize agenda and assign pre-reading
Team selection • Based on the Problem Statement • Key roles – Leader, “Angel”, SMEs
9 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 17
Review Meeting - Overview • Presentations – by the project team,
any format, reusing existing materials • Questions by the review team, at any
time, moderated by the Leader or Angel
• Strengths, Issues, Observations recorded on index cards (Low Tech)
• Team Caucus determines findings • Readout and (Optional) Sponsor/
Client meeting
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 18
Paper-based reviews can be
hard to transcribe! FMD
{ Write large, One thought per card, with enough detail to be useful – but not too wordy.
Save for Sequence Number
Save for Severity
Note here if card is a Strength or Observation; default is an Issue
Don’t Forget Your Initials!
Snow Card Example - 1
10 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 19
Review Meeting - Caucus • Led by review Leader and Angel. • Categorization is done to provide a
consistent “story” for the read-out – A “typical” review produces 80–350 observations,
arranged into 10-25 topics
• Issues are Prioritized:“Management Alert”, Critical, Major, Minor
• Summary of each topic area written by team subject matter experts
• Strengths should be preserved if changes are made to the architecture
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 20
Prioritization by the Review Team Management Alert: • Goes to client/upper management • “In the unanimous opinion of the Review Team,
the project will fail unless these issues are immediately addressed: <List of issues>”
Critical, Major, Minor, Strengths, & Suggestions go to the project • Readout ensures no surprises in the report • The review team will NOT withdraw a finding or
lower the priority (project owns the action plan) • Priority may be increased at project request
11 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 21
Follow-up • Readout and Sponsor/ Client meeting • Management Alert and Critical Issue
Letters within a week Ø Most management alerts are related to cost,
schedule, and resources, NOT purely technical issues. Hence, upper management is responsible for the action plan for management alerts
• Critical Issue letters goes to the Project, Project management is responsible for the action plan
• Two Reports, one to the project, one to the Board.
• Review team Leader and Angel review the action plans.
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 22
What Happens When? • . . . a project ignores a Management Alert?
• . . . a project ignores a Critical issue? • . . . a project fixes everything? • . . . the review team identifies an issue that turns out to not be a problem (“false positives”)?
12 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 23
What's Different (or the Same) About this Kind of Review?
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 24
What's Different? • Focus – problem/solution congruence, Client ownership of
the decision making, ownership by project team • Context/ Domain independent – Notation and representation independent, not
model or standard based (although this can be included in the problem statement, if needed)
• Follow up – Clear statement of potential failure points, Project
sees and owns findings – can ignore or post on their web site
13 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 25
What's the Same? Outside eyes—external to team Products, processes, ideas are reviewed, not people Not a problem solving session Ø Balances a defined review
process with extreme flexibility.
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 26
Sponsor Survey - Benefits Saved money/time • Estimated at > $1M per large project, similar estimates
were derived from Monte Carlo simulations based on analyzing the impact from correcting review findings
• Direct results from specific reviews, e.g., re-architecting after the review trimmed 9 months off the schedule, saved > $7 million.
Intangible benefits include: • Preparation (especially preparing the problem
statement) forces people to think through the problem • Cross-pollination of techniques across the organization • Learning as a review team member • Synching up project teams; “on the same page”
14 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 27
Drawbacks to the Method • Risks of a Client-centered method
• The Client can grow out of touch with the market, customer needs, and end user issues
• In contracting, many of the success criteria are negotiated into the contract, and may represent political compromises that can’t be changed easily
• Method dependencies • Strongly oriented to face-to-face meetings, requires a
deep pool of reviewers with expertise
• Incomplete findings/level of detail • Any review’s findings are inherently incomplete. A high
level review is not suited for all problem spaces; 2 to 3 days will not flush out all of the “devils in the details”
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 28
Institutionalizing the Method • Requires building a “learning culture”
Executive Champion
Review Board
Review Organization
Project Project Project Project Project Project
Oversight
Reviewers, Reviews
Push Review Leaders
“Angels”
15 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 29
Suggested Steps for Making Reviews Part of the Culture • Start with a high-level champion/ executive sponsor • Obtain central funding for an extended period – say 5 years • Train a core team of leaders and angels to do reviews • Charter a board; recruit advocates. Board members are
expected to serve as review “Angels” • Perform the reviews; capture data and lessons learned to
document the method’s value • Move to a sponsor-based funding model after the initial
time period, when the method has proven itself • Rigorously retain the focus on client-centered problem
statements, project ownership of findings, limited but pertinent management alerts, and non-attribution
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 30
Questions???
16 Contextually Driven Architecture Reviews Dedolph – 10/18/2010
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 31
Presenter Info F. Michael Dedolph
412-477-7163 fmdedolph@netscape.net
BIO: F. Michael Dedolph is currently an innkeeper and owner of a bustling B&B, and part time consultant to the IT industry for reviews, risk management, quality methods, and process improvement. From 1997 to 2004, Michael was a systems architecture review leader at Bell Labs (Lucent); he also managed and taught Lucent’s Systems Architecture Introduction class. In addition to leading architecture Review Teams, he facilitated numerous risk identification, problem solving, and project retrospective sessions. Prior to 1997, Michael worked in the Risk and Process programs at the SEI. While at the SEI, he was the technical lead for the teams that developed the SCE and CBA-IPI appraisal methods, and was the team leader for several Risk Reviews. He started his IT career by spending 10 years as an Air Force computer officer, and also spent 7 years at CSC leading process improvement teams.
Better Software East Contextually Driven Architecture Reviews – Dedolph 11/14/2013 32
References: 1. CMU/SEI-2000-TR-004, "ATAM: Method for Architecture Evaluation";
Kazman, Klein, Clements, 2000 2. Problem Seeking: An Architectural Programming Primer, 4th Ed;
Pena and Parshall, 2001, ISBN 0-913962-87-2 3. Handbook of Walkthroughs, Inspections, and Technical Reviews;
Freedman and Weinberg, 1990, ISBN 0-932633-19-6 4. IEEE Software, March/April 2005, "Architecture Reviews: Practice and Experience";
Maranzano, Rozsypal, Zimmerman, Warnken, Wirth, and Weiss 5. STQE Jul/Aug 2002 (Vol. 4, Issue 4) Feature: Measurement & Analysis
"A Blueprint for Success: Implementing an architectural review system“; Daniel Starr, Gus Zimmerman
6. CMU/SEI-2006-TR-012, "Risk Themes Discovered Through Architecture Evaluations"; Bass, Nord, Wood, Zubrow; 2006
7. CMU/SEI-2010-TN-018, “Relating Business Goals to Architecturally Significant Requirements for Software Systems”; Clements, Bass; 2010
8. CMU/SEI Webinar, “SoS Architecture Evaluation and Quality Attribute Specification (Webinar)”; Gagliardi; 2010