Architecture Architecture Review ProcessesReview Processes

Architecture interaction in Architecture interaction in project oversight and project oversight and

execution.execution.

What’s Architecture?What’s Architecture?

ArchitectureArchitecture The fundamental organization of a system, The fundamental organization of a system,

embodied in its components, their embodied in its components, their relationships to each other and the relationships to each other and the environment, and the principles governing environment, and the principles governing its design and evolution.its design and evolution.

ISO/IEC 42010:2007ISO/IEC 42010:2007

A formal description of a system, or a A formal description of a system, or a detailed plan of the system at component detailed plan of the system at component level to guide its implementation.level to guide its implementation.

TOGAFTOGAF

The structure of components, their inter-The structure of components, their inter-relationships, and the principles and relationships, and the principles and guidelines governing their design and guidelines governing their design and evolution over time. evolution over time.

TOGAFTOGAF

Things That Architects Things That Architects Don’t DoDon’t Do

Process compliance – that’s project officeProcess compliance – that’s project office Estimates – that’s service ownersEstimates – that’s service owners Hardware specifications – that’s Hardware specifications – that’s

EngineeringEngineering Software specifications – that’s DevelopersSoftware specifications – that’s Developers Requirement gathering – that’s Business Requirement gathering – that’s Business

AnalystsAnalysts Process description – that’s Business Process description – that’s Business

AnalystsAnalysts Windows – that’s Building MaintenanceWindows – that’s Building Maintenance

Things That Architects Things That Architects Can DoCan Do

Plan technology direction and set technology standardsPlan technology direction and set technology standards Help you figure out which technologies you should support.Help you figure out which technologies you should support.

Review plans, designs and purchasesReview plans, designs and purchases Assess how well a plan aligns with current direction and desired Assess how well a plan aligns with current direction and desired

future positions.future positions. Identify opportunities to reuse components and services.Identify opportunities to reuse components and services.

Leverage enterprise contracts and license agreements.Leverage enterprise contracts and license agreements. Integrate shared services where they might be cost-effective.Integrate shared services where they might be cost-effective.

Review business organization and business processesReview business organization and business processes Technical Architecture: align your technology plan with Technical Architecture: align your technology plan with

enterprise goals, business plans and business processes. enterprise goals, business plans and business processes. Enterprise Architecture: align your business plans, business Enterprise Architecture: align your business plans, business

process and technology plan with your enterprise goals.process and technology plan with your enterprise goals.

Architecture ReviewsArchitecture Reviews

You are about to sign off on the software You are about to sign off on the software architecture of a multimillion-dollar software-architecture of a multimillion-dollar software-intensive system. intensive system.

What assurance do you have that the What assurance do you have that the architectural decisions underpinning the architectural decisions underpinning the design are the right ones to deliver a system design are the right ones to deliver a system that meets the required business goals? that meets the required business goals?

Source: Software Architecture Review: The State of Practice Muhammad Ali Babar, Lero—the Irish Software Engineering Research Centre Source: Software Architecture Review: The State of Practice Muhammad Ali Babar, Lero—the Irish Software Engineering Research Centre Ian Gorton, Pacific Northwest National Laboratory computer COMPUTING PRACTICES Published by the IEEE Computer Society Ian Gorton, Pacific Northwest National Laboratory computer COMPUTING PRACTICES Published by the IEEE Computer Society 0018-9162/09/$25.00 © 2009 IEEE0018-9162/09/$25.00 © 2009 IEEE

Architecture ReviewsArchitecture Reviews

How sure are you that the project will not be How sure are you that the project will not be delayed through downstream rework—or delayed through downstream rework—or even fail—due to inappropriate architectural even fail—due to inappropriate architectural choices?choices?

Do you know whether all system Do you know whether all system stakeholders have confidence in the stakeholders have confidence in the proposed solution? proposed solution?

Are “best endeavors” a good enough basis Are “best endeavors” a good enough basis for accepting an architectural design?for accepting an architectural design?

Source: Software Architecture Review: The State of Practice Muhammad Ali Babar, Lero—the Irish Software Engineering Source: Software Architecture Review: The State of Practice Muhammad Ali Babar, Lero—the Irish Software Engineering Research Centre Ian Gorton, Pacific Northwest National Laboratory computer COMPUTING PRACTICES Published by Research Centre Ian Gorton, Pacific Northwest National Laboratory computer COMPUTING PRACTICES Published by the IEEE Computer Society 0018-9162/09/$25.00 © 2009 IEEEthe IEEE Computer Society 0018-9162/09/$25.00 © 2009 IEEE

Architecture ReviewsArchitecture Reviews Architecture reviews are an effective way of Architecture reviews are an effective way of

ensuring design quality and addressing ensuring design quality and addressing architectural concerns. architectural concerns.

The principal objectives of a software The principal objectives of a software architecture review are to assess an architecture review are to assess an architecture’s ability to deliver a system capable architecture’s ability to deliver a system capable of fulfilling the quality requirements and to of fulfilling the quality requirements and to identify potential risks.identify potential risks.11

1.1. P. Clements, R. Kazman, and M. Klein, Evaluating Software Architectures: Methods and Case Studies, Addison-Wesley, 2002. P. Clements, R. Kazman, and M. Klein, Evaluating Software Architectures: Methods and Case Studies, Addison-Wesley, 2002.

Source: Software Architecture Review: The State of Practice Muhammad Ali Babar, Lero—the Irish Software Engineering Research Centre Source: Software Architecture Review: The State of Practice Muhammad Ali Babar, Lero—the Irish Software Engineering Research Centre Ian Gorton, Pacific Northwest National Laboratory computer COMPUTING PRACTICES Published by the IEEE Computer Society 0018-Ian Gorton, Pacific Northwest National Laboratory computer COMPUTING PRACTICES Published by the IEEE Computer Society 0018-9162/09/$25.00 © 2009 IEEE9162/09/$25.00 © 2009 IEEE

Benefits of Architecture Benefits of Architecture ReviewReview

Identifying potential risks in the proposed architecture (88%)Identifying potential risks in the proposed architecture (88%) Assessing quality attributes (for example, scalability, performance) Assessing quality attributes (for example, scalability, performance)

(77%)(77%) Identifying opportunities for reuse of artifacts and components Identifying opportunities for reuse of artifacts and components

(72%)(72%) Promoting good architecture design and evaluation practices (64%)Promoting good architecture design and evaluation practices (64%) Reducing project cost caused by undetected design problems (63%)Reducing project cost caused by undetected design problems (63%) Capturing the rationale for important design decisions (59%)Capturing the rationale for important design decisions (59%) Uncovering problems and conflicts in requirements (59%)Uncovering problems and conflicts in requirements (59%) Conforming to organization’s quality assurance process (55%)Conforming to organization’s quality assurance process (55%) Assisting stakeholders in negotiating conflicting requirements (43%)Assisting stakeholders in negotiating conflicting requirements (43%) Partitioning architectural design responsibilities (40%)Partitioning architectural design responsibilities (40%) Identifying skills required to implement the proposed architecture Identifying skills required to implement the proposed architecture

(40%)(40%) Improving architecture documentation quality (40%)Improving architecture documentation quality (40%) Facilitating clear articulation of nonfunctional requirements (31%)Facilitating clear articulation of nonfunctional requirements (31%) Opening new communication channels among stakeholders (31%)Opening new communication channels among stakeholders (31%)

Survey of benefits perceived from systems architecture among 86 responding organizations, from Survey of benefits perceived from systems architecture among 86 responding organizations, from Software Architecture Review: The State of Software Architecture Review: The State of Practice by Practice by Muhammad Ali Babar, Muhammad Ali Babar, Lero—the Irish Software Engineering Research Centre and Lero—the Irish Software Engineering Research Centre and Ian Gorton, Ian Gorton, Pacific Northwest National Pacific Northwest National Laboratory, IEEE Computer July 2009 © IEEE 2009Laboratory, IEEE Computer July 2009 © IEEE 2009

Types of reviewTypes of review Project process reviewsProject process reviews

Project Initiation Review (Gate 1)Project Initiation Review (Gate 1) Approve project goals, strategy, conceptApprove project goals, strategy, concept Iterative projects may propose how they will articulate Iterative projects may propose how they will articulate

architecture and designarchitecture and design Planning / Design Review (Gate 2)Planning / Design Review (Gate 2)

Approve project architecture, solution design, technology Approve project architecture, solution design, technology directiondirection

Do this each time architecture changesDo this each time architecture changes Execution / Build / Pilot Review (pre-release) (Gate 3)Execution / Build / Pilot Review (pre-release) (Gate 3)

Approve architecture /design changes that may occur during Approve architecture /design changes that may occur during E&BE&B

Purchase process reviewsPurchase process reviews Pre-purchase Review (RFP, IFB)Pre-purchase Review (RFP, IFB)

Ensure sensible technical language in requirementsEnsure sensible technical language in requirements Purchase Proposal Review (pre-award)Purchase Proposal Review (pre-award)

Approve technology selections, architecture and strategy of Approve technology selections, architecture and strategy of proposalproposal

Basic review flowBasic review flow Submit documents (project Submit documents (project

team)team) Review documents (architect)Review documents (architect) If issues are found:If issues are found:

Resolve issuesResolve issues Re-submitRe-submit

If issues are not resolved:If issues are not resolved: Approve with issue or RejectApprove with issue or Reject

If rejected:If rejected: Re-plan and resubmit or haltRe-plan and resubmit or halt

If approved with issueIf approved with issue Track and resolve issue later onTrack and resolve issue later on

start

Read all supplied documentation

TASDPPM Business

CasePurchasing Plan

Other documents supplied

Is there a concern not

addressed by the agency?

Inquire about it – why does it appear inconsistent, what can mitigate the issue, or what do I

misunderstand?

Questions, issues, comments and/or

concerns document

Formulate reply and/or fix documentation

Y

N, or all addressed previously

Is there an open issue or

risk?

Document Issues or Risks in PPM for resolution

Can we approve with Issues/Risks?

Mitigate Issue/Risk

Y

N

Y

Recommend approval, sign off

PPMY

Can we wait for mitigation?

N

Recommend rejection, reject in

PPM

N

End

Research raised issues, to discover

context and determine good

practice expectations

Comment Replies and revised design

documents

8/19/2009ITS Architecture Don Jerman

A&E Interaction for typical architecture review.

Legend

Agency Document prepared prior to

review (typically in PPM)

A&E Standard process

A&E process step for resolution of

concerns.

Agency process step for resolution

of concerns

Architecture Views and Architecture Views and PerspectivesPerspectives

ViewpointsViewpoints FunctionalFunctional

What does it need to do?What does it need to do? InformationInformation

What goes in, comes out, and What goes in, comes out, and what do we keep?what do we keep?

ConcurrencyConcurrency How do the parts work How do the parts work

together, at the same time?together, at the same time? DevelopmentDevelopment

How will we produce the How will we produce the system?system?

DeploymentDeployment How will we deliver it?How will we deliver it? How will we retire the old How will we retire the old

system?system? OperationOperation

How will we keep it going?How will we keep it going? When will we stop it?When will we stop it?

PerspectivesPerspectives Accessibility and UsabilityAccessibility and Usability

Will people be able to use it?Will people be able to use it? Availability and Resilience Availability and Resilience

Will it work when it needs to?Will it work when it needs to? Regulation Regulation

What are we required to do?What are we required to do? SecuritySecurity

Are we protecting it?Are we protecting it? Development Resource Development Resource

What will it take to do it?What will it take to do it? Evolution Evolution

How will it grow and change?How will it grow and change? Location Location

Where will we build it?Where will we build it? Performance and Scalability Performance and Scalability

Will it handle all the work?Will it handle all the work?

Project Reviews are Unique Project Reviews are Unique EffortsEfforts

We do not specify a set of criteria that can be applied to any project, because the concerns and goals of each IT project are unique.

We do follow a pattern and a set of guidelines that inform our review, and use experience-based reasoning to evaluate projects.

This is reflective of the state of current architecture practice in the industry. “When asked about the nature of the architecture review

process, 56 percent of respondents described their organization’s review process as informal, 41 percent reported a formal process in place, and 3 percent were not sure, as architecture review processes varied from project to project.”*

“The two most common techniques applied to review an architecture are experience-based reasoning (83 percent) and prototyping (70 percent)”*

*From *From Software Architecture Review: The State of Practice by Software Architecture Review: The State of Practice by Muhammad Ali Babar, Muhammad Ali Babar, Lero—the Irish Software Lero—the Irish Software Engineering Research Centre and Engineering Research Centre and Ian Gorton, Ian Gorton, Pacific Northwest National Laboratory, IEEE Computer July 2009 Pacific Northwest National Laboratory, IEEE Computer July 2009 © IEEE 2009© IEEE 2009

Better review flow for Better review flow for complex projectscomplex projects

Engage architects before the gate reviewEngage architects before the gate review Initiation discussions: Initiation discussions: determine strategy determine strategy

optionsoptions Early planning option: conceptual discussionsEarly planning option: conceptual discussions Mid-planning option: design decision discussionsMid-planning option: design decision discussions

Obtain guidance prior to reviewObtain guidance prior to review Get pre-reviews as effort is sunk in the TASD.Get pre-reviews as effort is sunk in the TASD. Guide the design effort with the results of pre-Guide the design effort with the results of pre-

reviews.reviews. Balance review frequency based on risk and Balance review frequency based on risk and

remaining effort.remaining effort. Don’t arrive at the gate with a big gapDon’t arrive at the gate with a big gap

Closing a gap between what you have and what’s Closing a gap between what you have and what’s required will take up time and effort.required will take up time and effort.

Discovering the gap Discovering the gap at the gateat the gate extends the extends the project.project.

Small corrections during the development Small corrections during the development process makes a more manageable schedule and process makes a more manageable schedule and allows approval reviews to go faster, too.allows approval reviews to go faster, too.

?

Pipeline Vs BottleneckPipeline Vs Bottleneck

Work architecture activities Work architecture activities into the project plan.into the project plan.

Schedule time to pre-review Schedule time to pre-review and refine architecture.and refine architecture.

Balance work-at-risk with Balance work-at-risk with review effort and resources.review effort and resources.

Plan architecture updates Plan architecture updates into project change process.into project change process.

Arrive at gates ready-to-pass.Arrive at gates ready-to-pass.

Treat architecture activities Treat architecture activities as “externalities” to the plan.as “externalities” to the plan.

Assign architecture Assign architecture document production to non-document production to non-technical contributors.technical contributors.

Plan short reviews of Plan short reviews of documents that take a long documents that take a long time to write. time to write.

Connect with the Connect with the architecture group only at architecture group only at gate reviews.gate reviews.

Blame the process for gate Blame the process for gate delays.delays.

Pipeline Strategies Bottleneck Strategies

Pipeline strategies keep project progress flowing smoothly by planning for review. Bottleneck strategies ensure schedule overruns and risk re-work on design artifacts.

Minimum Review Minimum Review RequirementsRequirements

Gate 1Gate 1 PPM Project Info TabPPM Project Info Tab Agency ApprovalAgency Approval (Registration projects may have to (Registration projects may have to

deliver more information because this deliver more information because this is the only review)is the only review)

Gate 2Gate 2 Complete TASD (all Complete TASD (all

parts)parts) Complete DesignComplete Design Complete Execution Complete Execution

PlanPlan Procurement Plan (if Procurement Plan (if

applicable)applicable) Agency ApprovalAgency Approval

Gate 3Gate 3 Updated TASD and Updated TASD and

documents (all project documents (all project changes reflected in changes reflected in documentation).documentation).

Release planRelease plan Operation planOperation plan Agency ApprovalAgency Approval

Pre-purchasePre-purchase Coherent requirementsCoherent requirements Architecture disclosure Architecture disclosure

and conformance and conformance requiredrequired

Pre-AwardPre-Award Coherent and compliant Coherent and compliant

proposalproposal

Design ProcessDesign Process

Business Requirement Conceptual Architecture

Logical Design

Implementation

Architecture

Engineering

Preliminary Architecture

EngineeringDesign

Gate1

Gate 2

Gate 3

Architecture Architecture documentationdocumentation

PPM Project InformationPPM Project Information Establish project strategy, goals and scopeEstablish project strategy, goals and scope State assumptions and constraintsState assumptions and constraints Establish key deliverables and milestonesEstablish key deliverables and milestones Collect project documents and design artifactsCollect project documents and design artifacts

Technical Architecture System Design Technical Architecture System Design TemplateTemplate Functional Diagram & NarrativeFunctional Diagram & Narrative Conceptual Diagram, Narrative & ChecklistConceptual Diagram, Narrative & Checklist Preliminary Diagram, Narrative & ChecklistPreliminary Diagram, Narrative & Checklist Detail Diagram, Narrative & ChecklistDetail Diagram, Narrative & Checklist

Purchasing PlanPurchasing Plan Establish strategy and provide overview context Establish strategy and provide overview context

for planned purchasesfor planned purchases

PPM and ArchitecturePPM and Architecture Project Info TabProject Info Tab

Stakeholders, Contacts and DatesStakeholders, Contacts and Dates Business Case, Scope and GoalsBusiness Case, Scope and Goals Strategy for project executionStrategy for project execution Assumptions & ConstraintsAssumptions & Constraints

Issues and Risks TabIssues and Risks Tab Track issues that the team has surfacedTrack issues that the team has surfaced Track issues that Architects surfaceTrack issues that Architects surface

Document ManagementDocument Management Collect TASD, Procurement Plan, and other Collect TASD, Procurement Plan, and other

artifacts.artifacts.

TASD TemplateTASD Template It’s a templateIt’s a template

It’s the official document for communicating architecture, and It’s the official document for communicating architecture, and the format has been formally approvedthe format has been formally approved

Take a uniform approach to presentation to reduce review Take a uniform approach to presentation to reduce review durations, put any extra information at the end of a sectiondurations, put any extra information at the end of a section

Follow the structure and leave the section headers intact, Follow the structure and leave the section headers intact, don’t delete things or you’ll confuse the reviewerdon’t delete things or you’ll confuse the reviewer

Obtain information to answer the questions where it is Obtain information to answer the questions where it is applicable but unknownapplicable but unknown

It’s a template, not a final documentIt’s a template, not a final document Add diagrams or checklist sections that are useful to telling Add diagrams or checklist sections that are useful to telling

the architecture storythe architecture story If desired, add agency-specific or project-specific sections at If desired, add agency-specific or project-specific sections at

the end of the document.the end of the document. Sample diagrams indicate the desired level of detail but are Sample diagrams indicate the desired level of detail but are

not intended to prescribe a formatnot intended to prescribe a format For very large or complex projects, alternative documents are For very large or complex projects, alternative documents are

possible, but they should be approved at initiationpossible, but they should be approved at initiation

TASD ContentTASD Content The checklists collect the most basic informationThe checklists collect the most basic information

They cover most of the important perspectives and points of view.They cover most of the important perspectives and points of view. They ensure the project team has considered some important design They ensure the project team has considered some important design

features.features. Architects can get a pretty good idea of a system by reading the Architects can get a pretty good idea of a system by reading the

checklist.checklist. Diagrams provide a roadmapDiagrams provide a roadmap

Relationships and connectivity that are difficult to narrate can be drawn Relationships and connectivity that are difficult to narrate can be drawn clearly.clearly.

Diagrams should be specific to the project and reflect important design Diagrams should be specific to the project and reflect important design features.features.

More than one diagram may be needed, particularly to represent More than one diagram may be needed, particularly to represent different viewpoints, or dynamic aspects of the system.different viewpoints, or dynamic aspects of the system.

Narratives complete the pictureNarratives complete the picture Not limited to the checklist topics, Narratives can fill in gaps in the Not limited to the checklist topics, Narratives can fill in gaps in the

TASD format.TASD format. Narratives can tell a story about data flow, about business interactions, Narratives can tell a story about data flow, about business interactions,

nonfunctional requirements, or about system relationships in more nonfunctional requirements, or about system relationships in more detail than a diagram can represent.detail than a diagram can represent.

Narratives can relate the decision process leading to design choices.Narratives can relate the decision process leading to design choices.

Architecture ResourcesArchitecture Resources OSCIO Architecture TeamOSCIO Architecture Team

Provide Statewide Architecture review and Provide Statewide Architecture review and interpretation.interpretation.

Agency Architect or Technical OfficerAgency Architect or Technical Officer Specialize in agency technology and business Specialize in agency technology and business

applications.applications. Formulate agency standards and practices.Formulate agency standards and practices.

Project Analysts and Technical TeamProject Analysts and Technical Team Bring expert knowledge about what the project Bring expert knowledge about what the project

intends to accomplish.intends to accomplish. Apply architecture standards to technical planning Apply architecture standards to technical planning

and implementation.and implementation. Design software and other technical components Design software and other technical components

to fit the architecture.to fit the architecture.

Avoiding Architecture Avoiding Architecture RiskRisk

Consider the architecture documentation Consider the architecture documentation risky product, and plan benchmark reviews risky product, and plan benchmark reviews that balance the effort sunk versus effort that balance the effort sunk versus effort remaining, and the risk of re-doing progress remaining, and the risk of re-doing progress since the last benchmark. since the last benchmark.

Don’t wait until the gate to start the TASD, Don’t wait until the gate to start the TASD, nor to find out whether it’s deficient.nor to find out whether it’s deficient.

Get a review of detailed architecture design Get a review of detailed architecture design before engineering the detail components.before engineering the detail components.

As PM or PMA, make sure that business As PM or PMA, make sure that business analysts, architects and engineers all update analysts, architects and engineers all update their part of the architecture documentation their part of the architecture documentation when altering their detail designs.when altering their detail designs.

Architecture Architecture Review ProcessesReview Processes

Architecture interaction in Architecture interaction in project oversight and project oversight and

execution.execution.

Architecture ReviewArchitecture Review

We perform architecture reviews to ensure: We perform architecture reviews to ensure: The architecture of a system is documented.The architecture of a system is documented. It provides a coherent description of the It provides a coherent description of the

system.system. It is conformant to State and Agency It is conformant to State and Agency

principles, standards and plans.principles, standards and plans. It is compatible with the legacy technical It is compatible with the legacy technical

landscape.landscape. That the chosen technology and design is That the chosen technology and design is

likely to achieve the project’s goals and likely to achieve the project’s goals and objectives.objectives.

Office of the State CIOOffice of the State CIOStrategic Planning OfficeStrategic Planning Office

Architecture TeamArchitecture Team Doug Banich 919-754-6210Doug Banich 919-754-6210 douglas.banich@its.nc.govdouglas.banich@its.nc.gov