Lean & AgileSystems Engineeringfor Systems of Systems
Dr. David F. Rico, PMP, CSMWebsite: http://davidfrico.com
LinkedIn: http://www.linkedin.com/in/davidfricoFacebook: http://www.facebook.com/profile.php?id=1540017424
2
Agenda
IntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
3
AuthorDoD contractor with 25+ years of IT experienceB.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys.Large gov’t projects in U.S., Far/Mid-East, & Europe
Published six books & numerous journal articlesExpertise in metrics, models, & cost engineeringAdjunct at George Washington, UMUC, & ArgosySix Sigma, CMMI, ISO 9001, DoDAF & DoD 5000Agile Program Management & Lean Development
4
Purpose of BriefingProvide an overview of traditional, lean, and agile systems engineering concepts:
Define systems engineering, its purpose, and identify major approaches to traditional systems developmentIdentify the strengths and weaknesses of traditional systems engineering for today’s ever changing worldDiscuss lean and agile systems engineering as a means of managing ever increasing system complexityIntroduce mechanisms for scaling lean and agile systems engineering for larger systems of systemsExamine iterative testing techniques within agile systems engineering for verification and validation
5
AgendaIntroduction
Systems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
6
What is Systems Engineering?Sys-tem (sĭs-'təm): Interacting, interrelated, interdependent elements; A complex whole
Interdisciplinary approach and means to enable the realization of successful systems [INCOSE]Interdisciplinary tasks required to transformcustomer needs into a system solution [IEEE]Interdisciplinary approach for transforming a set of customer needs into a product solution [CMMI]Interdisciplinary approach for translating mission needs into operational systems [DoD 5000]Interdisciplinary processes spanning the conceptionof ideas through the retirement of a system [ISO]
7
Purpose of Systems EngineeringManage increasing system complexity (1950s)Optimize [sub]system performance (1960s)Improve system cost and quality (1970s)
Eisner, H. (2002). Essentials of project and systems engineering management. New York, NY: John Wiley & Sons.Blanchard, B. S., & Fabrycky, W. J. (2006). Systems engineering and analysis. Upper Saddle River, NJ: Pearson Prentice-Hall.
8
MIL-STD-1521BCreated by U.S. Air Force in 1976Framework for system and software reviewsStandardized milestone reviews and technical audits
U.S. Department of Defense. (1985). Military standard: Technical reviews and audits for systems, equipments, and computer software (MIL-STD-1521B). Washington, DC: Air Force Systems Command (AFSC).
Note: ActualTime Framesof Activities
Must BeTailored to
Each Program
Test
DraftSystem
RequirementsSpecification
SystemRequirementsSpecification
HWCIDevelopmentSpecification
SoftwareTop Level
DesignDocument
SoftwareListing
HWCIProduct
Specification
SoftwareProduct
Specification
IRSDBDD
DraftHWCIProduct
Specification
Sourceand
ObjectCode
SystemRequirements
Review
SystemDesignReview Software
SpecificationReview
PreliminaryDesignReviewCSCI
PreliminaryDesignReviewHWCI
CriticalDesignReviewCSCI
CriticalDesignReviewHWCI
TestReadiness
Review
FunctionalConfiguration
AuditHWCI
PhysicalConfiguration
AuditHWCI
FormalQualification
ReviewHWCI
FunctionalConfiguration
AuditCSCI
PhysicalConfiguration
AuditCSCI
SystemFunctional
ConfigurationAudit
SystemPhysical
ConfigurationAudit
SystemFormal
QualificationReview
Prepare Testand
EvaluationMaster Plan
PrepareSoftware
TestPlan
PrepareHWCITestPlan
PrepareSoftware
TestDescription
PrepareHWCITest
Procedures
PrepareSoftware
TestProcedures
PerformHWCI
SubsystemTests
PerformSoftware
Tests
PrepareSoftware
TestReports
PerformSystemTests
PrepareSystem
TestReports
ProgramActivity
ConceptExploration
Phase
Demonstrationand Validation
PhaseFull Scale Development Phase
TechnicalReviews
andAudits
Specifica-tions and
otherProducts
SystemFunctional
Identification
AllocatedIdentification
Developmental Configuration(HWCI Only) Product
IdentificationSystem
FunctionalBaseline
AllocatedBaseline
ProductBaseline
SoftwareRequirementsSpecification
IRDSoftwareDetailedDesign
Document
9
MIL-STD-498Created by U.S. Navy in 1994Consolidated multiple U.S. DoD standardsSoftware process and documentation standard
U.S. Department of Defense. (1994). Military standard: Software development and documentation (MIL-STD-498). Arlington, VA: Space and Naval Warfare Center (SPAWAR).
10
ISO-15288Created by ISO/IEC around 2002Standardization of international practicesMeant for complex, computer-based systems
International Organization for Standardization/International Electrotechnical Commission. (2002). Standard for systems engineering: System life cycle processes (ISO/IEC 15288). Geneva, Switzerland: Author.
11
CMMICreated by the SEI in 2002Merger of SW-CMM, SA-CMM, IPD-CMM, etc.Used for systems engineering process improvement
CMMI Product Team. (2006). CMMI for development version 1.2 (CMU/SEI-2006-TR-008). Pittsburg, PA: Software Engineering Institute.
12
DoD Acquisition LifecycleCreated by the U.S. DoD around 2003Latest evolution of acquisition best practicesMeant for large-scale, multi-billion weapon systems
DAU. (2009). Integrated defense acquisition, technology, and logistics life cycle management framework. Retrieved October 9, 2009, from https://acc.dau.mil/ifc
13
Systems Engineering BenefitsStudy funded by Australian defense instituteAlmost 44 programs studied from 2001 to 2004Systems engineering minimizes schedule overruns
Honour, Eric C. (2009). Demographics in measuring systems engineering return on investment (SE-ROI). Proceedings of the Joint 19th Annual International Symposium of INCOSE/Third Asia-Pacific Conference on Systems Engineering (INCOSE/APCOSE 2009), Singapore.
14
Systems Engineering StudiesU.S. Air Force Center for Systems EngineeringCase studies of 9 major U.S.Air Force ProgramsPrograms had significant cost and technical issues
Air Force Institute of Technology (AFIT). (2009). Systems engineering case studies. Retrieved October 19, 2009, from http://www.afit.edu/cse/cases.cfm
PROGRAM NAME EST. COST EST. UNIT ACT. COST ACT. UNIT OVERRUN OTHER ISSUES
A-10 Thunderbolt $2,547,600,000 439 $5,511,490,000 360 21,005% Wing Reliability
B-2 Spirit $58,200,000,000 132 $45,300,000,000 20 57,949% Low Observability
C-5A Galaxy $3,413,200,000 120 $4,426,400,000 81 7,585% Maintenance
F-111 Aardvark $2,171,590,000 547 $8,652,000,000 547 298% Structural
RQ-4A Global Hawk $2,904,600,000 51 $3,816,700,000 51 31% Quality
GPS Navstar $814,400,000 28 $8,650,000,000 14 31,764% Sofware Reliability
HST Hubble $651,200,000 1 $1,458,900,000 1 124% Schedule, Mirror
KC-10 Extender $1,055,000,000 20 $1,090,600,000 20 3% Reliability
LGM-118A Peacekeeper $16,600,000,000 21 $16,600,000,000 21 0% Range
$9,817,510,000 151 $10,611,787,778 124 13,196% $85,655,686
15
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
16
What is a Challenge?Chal-lenge (chăl-'ənj): Contest, competition, fight, defy, confront, or dispute; To question
21st century systems are more software-intensiveand highly-complex with numerous invisible partsTechnology is evolving at an exponential rate of change which severely limits the planning horizonGlobal competitiveness has intensified and new military threats are rapidly emerging all of the timeCustomers have unpredictable needs and necessitate decision-making flexibility throughout the programToday’s post-industrial information age knowledge workers need new systems engineering approaches
17
Information AgeU.S. is no longer an industrial-age nationU.S. part of a group of post-industrial countriesU.S. consists of information-age knowledge workers
Bell, D. (1999). The coming of post industrial society. New York, NY: Basic Books.
0%
20%
40%
60%
80%
100%
1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990
Perc
ent o
f Eco
nom
y
Information
Service
Industry
Agriculture
18
System Complexity is Growing21st century systems are becoming more complexNumber of physical parts are becoming smallerNano-circuitry and software hide complexity
Moody, J. A., et al. (1997). Metrics and case studies for evaluating engineering designs. Upper Saddle River, NJ: Prentice-Hall.
19
Software CenturyNumber of software-intensive systems is growingYearly software industry revenue exceeds $3 trillionPoor software quality costing trillions in lost revenues
Dvorak, D. L. (2009). NASA study on flight software complexity. Pasadena, CA: Jet Propulsion Laboratory (JPL).
Software in Military Aircraft
8% 10%20%
35%45%
65%
80%
0%
20%
40%
60%
80%
100%
1960F-4
1964A-7
1970F-111
1975F-15
1982F-16
1990B-2
2000F-22
Perc
ent o
f Fun
ctio
nalit
y
20
Exponential Rate of ChangeTechnology evolving at an ever increasing rateNano-scale computers will become the norm soonTechnological breakthroughs may climax in 25 years
Kurzweil, R. (2005). The singularity is near: When humans transcend biology. New York, NY: Penguin Group.
21
Crossing the ChasmNew technology spreads very slowlyThere are a few innovators and early adoptersYears and decades for most to adopt new technology
Moore, G. A. (1991). Crossing the chasm: Marketing and selling technology to mainstream customers. New York, NY: Harper Business.
22
Coping With Big ChangesHumans can’t cope with large technological changeChanges may be resisted for a long time (years)Big projects plunge organizations into chaos
Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).
23
Global Market CompetitionGlobalization has intensified market competitionDomestic competition is no longer the major threatThe trade deficit with the Far East is growing bigger
24
Cyber Threats are GrowingCyber threats have increased 10-fold in last decade70% of cyber incidents perpetrated by U.S. citizensCyber threats coming from Far East less than 3%
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 20030
500
1000
1500
2000
2500
3000
3500
4000
4500
Incidents
Vulnerablities
25
Complex Systems are UnstableLarge systems experience big downstream changesProject plans designed to cope with small changesSystems engineering not well-suited for changes
Jones, C. (1995). Patterns of software system failure and success. Boston, MA: International Thompson Computer Press.
Basic Assembly 575JCL 400Macro Assembly 400C 225Cobol 74 (Cobol I) 220FORTRAN 210Cobol 85 (Cobol II) 175Pascal 160PL/1 126RPG I 120RPG II/III 110Natural 100C++ 80Java 80dBase III 60Focus 60Clipper 60Oracle 60Sybase 60dBase IV 55Perl 50JavaScript 50VBScript 50Shell Script 50SAS 50APL 50
26
High Project Failure RatesFailed and challenged projects hover around 70%High failure rate due to inability to cope with changeBig projects exacerbate challenge and failure potential
Johnson, J., et al. (2009). Chaos summary 2009. Boston, MA: Standish Group International.
27
AgendaIntroductionSystems EngineeringSystems Engineering Challenges
Lean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
28
What is Lean?Lean (lēn): Thin, slim, slender, narrow, adequate, or just-enough; Without waste
A customer-driven systems engineering process that delivers the maximum amount of business valueAn economical systems engineering way of planning and managing the development of complex systemsA systems engineering process that is free of excess waste, capacity, and non-value adding activitiesJust-enough, just-in-time, and right-sized systems engineering processes, documentation, and toolsA systems engineering approach that is adaptable to change in customer needs and market conditions
29
Lean ThinkingTerm coined by John Krafcik of MIT in 1988Taiichi Ohno of Toyota is credited with its ideasToyota Production System was adapted from Ford
Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.
30
Lean Six SigmaCreated in late 1990s by Allied Signal and MaytagCombination of Six Sigma and Lean ThinkingFocuses on eliminating waste vs. variation
George, M. L. (2002). Lean six sigma: Combining six sigma quality with lean speed. New York, NY: McGraw-Hill.
31
Lean DevelopmentLean product development emerged in the 1980sAdaptation of Toyota Production System (TPS)“Toyota [New] Product Development System”
Clark, K. B., & Fujimoto, T. (1991). Product development performance: Strategy, organization, and management in the world auto industry. Boston, MA: Harvard Business School Press.
Lean Development
Frequent set-up changes
Lean Manufacturing
Short manufacturing throughput time
Reduced work-in-process inventory between manufacturing steps
Frequent transfer of small batches of parts between manufacturing steps
Reduced inventory requires slack resources and more information flow between steps
Adaptability to changes in volume, product mix, and product design
Broad task assignments for production workers gives higher productivity
Focus on quick problem solving and continuous process improvement
Simultaneous improvement in quality, delivery time, and manufacturing productivity
Frequent product changes
Short development time
Reduced information inventory between development steps
Frequent transfer of preliminary informationbetween development steps
Reduced development time requires slack resources and information flow between stages
Adaptability to changes in product design, schedule, and cost targets
Broad task assignments for engineers (developers) gives higher productivity
Focus on frequent incremental innovation and continuous product and process improvement
Simultaneous improvement in quality, development time, and development productivity
32
Lean Systems EngineeringOrigin in MIT Lean Aerospace Initiative in 1992Lean Systems Engineering WG formed in 2006Lean Enablers for Systems Engineering in 2009
INCOSE. (2009). Lean enablers for systems engineering. Retrieved October 20, 2009, from http://www.incose.org/practice/techactivities/wg/leansewg
33
Lean+ 10XCreated by Charles Toups of Boeing in 2008In-use by P-8A Poseidon and AEW&C SystemAdaptation of lean thinking for non-manufacturing
Brabant, E. M. (2009). Simple as. Retrieved October 20, 2009, from http://www.boeing.com/news/frontiers/i_ids01.pdf
34
Lean Engineering BenefitsMIT has studied dozens of systems for last 15 yearsThey applied criteria to determine if they were leanNumerous programs, past, present, and future
Murman, E., et al. (2002). Lean enterprise value: Insights from MIT's lean aerospace initiative. New York, NY: Palgrave.
35
AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems Engineering
Agile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
36
What is Agility?A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness, and ease of movement; To be very nimble
The ability to create and respond to change in order to profit in a turbulent global business environmentThe ability to quickly reprioritize use of resources when requirements, technology, and knowledge shiftA very fast response to sudden market changes and emerging threats by intensive customer interactionUse of evolutionary, incremental, and iterativedelivery to converge on an optimal customer solutionMaximizing the business value with right-sized, just-enough, and just-in-time processes and documentation
37
What are Agile Methods?‘Adaptable’ software development methodologies‘Human-centric’ method for creating business value‘Alternative’ to large document-based methodologies
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
38
Crystal MethodsCreated by Alistair Cockburn in 1991Has 14 practices, 10 roles, and 25 productsScalable family of techniques for critical systems
Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.
39
ScrumCreated by Jeff Sutherland at Easel in 1993Has 5 practices, 3 roles, 5 products, rules, etc.Uses EVM to burn down backlog in 30-day iterations
Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.
40
Dynamic Systems Develop.Created by group of British firms in 199315 practices, 12 roles, and 23 work productsNon-proprietary RAD approach from early 1990s
Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.
41
Feature Driven DevelopmentCreated by Jeff De Luca at Nebulon in 1997Has 8 practices, 14 roles, and 16 work productsUses object-oriented design and code inspections
Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.
42
Extreme ProgrammingCreated by Kent Beck at Chrysler in 1998Has 28 practices, 7 roles, and 7 work productsPopularized pair programming and test-driven dev.
Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.
43
Side-Effects of Agile MethodsEnable us to cross-the-chasm sooner or earlierReduce chaos associated with large-scale changeReduce or divide the risk of change into small pieces
Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).
44
Essence of Agile MethodsHigh degree of customer & developer interactionHighly-skilled teams producing frequent iterationsRight-sized, just-enough, and just-in-time process
Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
45
AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems Engineering
Agile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
46
What is a Practice?Prac-tice (prăk-'tĭs): Action, tool, technique, or work instruction; Step-by-step procedure
A set of one or more systems engineering techniquesto accomplish a specific action or desired outcomeStandard or semi-formal best practices or rules-of-thumb that are proven to be effective or efficientA suite of manual or automated tools or instruments that are useful for system design and developmentAn array of optional elements that may be employed on an as-needed basis, i.e., right tool at the right timeValue-adding action that may significantly enhance productivity, quality, or other key performance metric
47
Release PlanningCreated by Kent Beck at Chrysler in 1998Project plan with a 30-60-90-day timing horizonDisciplined and adaptable project management F/W
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
48
Onsite CustomersTerm coined by Kent Beck in 1999Customer who sits with developers full-timeFast and efficient way to capture customer needs
Tabaka, J. (2006). Collaboration explained: Facilitation skills for software project leaders. Upper Saddle River, NJ: Addison Wesley.
49
User StoriesTerm coined by Kent Beck in 1999Functions or features of value to customersHighly adaptable requirements engineering process
Cohn, M. (2004). User stories applied: For agile software development. Boston, MA: Addison-Wesley.
50
Pair ProgrammingTerm coined by Jim Coplien in 1995Consists of two side-by-side programmersHighly-effective group problem-solving technique
Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.
51
RefactoringTerm coined by William Opdyke in 1990Process of frequently rewriting source codeImproves readability, maintainability, and quality
Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.
52
Test-Driven DevelopmentTerm coined by Kent Beck in 2003Consists of writing all tests before codingEnsures all source code is verified and validated
Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.
53
Continuous IntegrationTerm coined by Martin Fowler in 1998Process of automated build/regression testingEvaluates impact of changes against entire system
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
BuildIntegration
Server
VersionControlServer
BuildScripts
UsesWatches
BuildStatus
ProvidesDeveloper A
Developer B
Developer C
CommitsChanges
CommitsChanges
CommitsChanges
Compile Source Code
Run Unit Tests
Run Coverage TestsStatic Code Analysis
Build Database
Generate Help FilesArchive and Deploy
54
Agile DocumentationMyth that voluminous documentation is neededMyth that agile methods do not use documentationRight-sized, just-in-time, and just enough documents
Rueping, A. (2003). Agile documentation: A pattern guide to producing lightweight documents for software projects. West Sussex, England: John Wiley & Sons.
Contracts
Document Type
Project Plans
Requirements
Architecture
Design
Coding
Tests
User guides
Quality Assurance
Agile Documentation
Performance-based, time-and-materials, level-of-effort
Release plans, iteration plans, story boards, agile repositories
User stories, wire frames, use cases, paper prototypes
Metaphors, spikes, system modeling language, DoDAF
Wire frames, design patterns, unified modeling language
Code patterns, program design language, coding comments
Unit, component, integration, system, and acceptance tests
XML documents, online help, Wikis, FAQs, video and audio clips
Performance, reliability, code structure analysis, and test reports
55
Which Practices Are In-UseSurveys of agile practices are conducted annuallyRelease planning is the most often used practiceContinuous integration is also a major practice
Version One. (2008). The state of agile development: Third annual survey. Alpharetta, GA: Author.
86%
75%
72%
65%
62%
60%
59%
59%
52%
49%
44%
35%
34%
31%
31%
0% 20% 40% 60% 80% 100%
Iteration planning
Daily standups
Release planning
Continuous integration
Automated builds
Burndown
Retrospectives
Refactoring
Velocity
Test-driven development
Open work area
Digital taskboard
On-site customer
Pair programming
Information radiators
56
AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering Practices
Agile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
57
What is Scalability?Scal-a-ble (skāl-'ə-bəl): To expand, grow, stretch, raise, or intensify; Increase in size
A systems engineering process that applies to projects of varying size, scope, magnitude, and complexityA product development process that is tailorable to the type, kind, and class of product under developmentA process that works well on range of products, from small to large programs involving systems of systemsAn approach that enables control of time, cost, scope, quality, and performance regardless of program typeSystems engineering processes designed to maximize business value under a wide variety of constraints
58
Multi-Level TeamsEnables projects to plan for the future and presentDecomposes capabilities into implementable piecesUnclogs the drainpipes to let the execution flow freely
Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.
59
Multi-Level BacklogEnables multiple levels of abstraction to co-existAllows customers and developers to communicateMakes optimum use of everyone’s time and resources
Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.
Feature SetRelated user stories that are grouped togetherAlso called a Theme, i.e., implemented as entityComprises 6 to 30 days worth of work
CapabilityHigh-level business or product functionAlso called an Epic, i.e., multiple feature setsComprises 18-90 days worth of work
User StorySimple requirement written by customer or userA small unit of functionality having business valueComprises 2 to 10 days worth of work
60
Multi-Level PlanningEnables multiple level enterprise plans to co-existAllows stakeholders to build viewpoint-specific plansEnsures capabilities are delivered at regular intervals
Release PlanFeature set focused, subsystem architectureStrategy, objectives, and backlog6 to 12 weeks
Product RoadmapCapability focused, enterprise architecture needsVision, objectives, backlog18 to 36 weeks
Iteration PlanUser story focused, component-level architectureImplementation plan, objectives, and backlog2 to 4 weeks
Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.
61
Multi-Level CoordinationEnables lean and agile methods to scale-upAllows enterprises to create large-scale programsUnleashes optimum productivity and overall control
Schwaber, K. (2004). Agile project management with scrum. Redmond, WA: Microsoft Press.
62
Multi-Level GovernanceEnables enterprises to achieve functional needsAllows programs to coordinate functional activitiesEnsures optimal technical performance is achieved
Ambler, S. W. (2009). Scaling agile software development through lean governance. Proceedings of the 2009 ICSE Workshop on Software Development Governance, Vancouver, Canada, 1-2.
Q
I
R
W
T
A
S
C
D
R
R
R
R
R
R
R
R
R
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
T
T
T
T
T
T
T
T
T
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
S
S
S
S
S
S
S
S
S
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
R T S
63
Multi-Level PMOEnables enterprises to optimize project performanceAllows enterprises to control and monitor programsEnsures projects are operating at peak capability
Augustine, S., & Cuellar, R. (2006). The lean-agile PMO: Using lean thinking to accelerate agile project delivery. Agile Project Management Executive Report, 7(10), 1-28.
64
Multi-Level AutomationEnables enterprises to be flexible but disciplinedAllows enterprises to distribute project work teamsEnsures distributed project teams are collaborating
Project Management
RequirementsDOORSRequisite ProSLATE
DesignRhapsodyTelelogic System ArchitectRational System Architect
CodingEclipseVisual StudioSun Studio
TestingJUnitNUnitXunitCPPUnit
GtestFitFitnesseSelenium
Quality AssuranceCheckStylePMDEMMAJdependCoberturaGcov
Configuration MgtSubversion (SVN)Concurrent Versions Sys.ClearCase
Build AutomationAntNAntMavenMake
Continuous Integ.Cruise ControlHudsonBuildBot
CollaborationWebExSkypeMeetMeWimba
WikiMediaWikiTracWikiPhpWiki
DocumentationNDocJavadocDoxygeniText
Version OneRallyScrum WorksVSTS
Agile TeamAgile EnterpriseScope ManagerStory Studio
XP Plan ItIterateXP TrackerAgilo
XP CGIXP WebXplannerIce Scrum
Project CardsTarget ProcessXtreme PlannerTeam System
CommunityEnterpriseMingleHansoft
65
AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering Scaling
Agile Systems Engineering TestingAgile Systems Engineering ValueSummary
66
What is Integration?In-te-gra-tion (ĭn-'tĭ-grā-shən): To add, group, mix, or assemble; Act of combining
A critical verification and validation step in the systems engineering process for a complex new systemA process of testing and evaluating units, components, subsystems, systems, and systems of systemsA key best practice that enables suppliers to deliver operational systems to customers early and oftenA automated systems development process that lowers the risks of developing large-scale complex systemsA lean and efficient process that maximizes business value by eliminating waste from traditional testing
67
What is Agile Testing?Traditional testing is a late, manual processAgile testing is an early and automated processThe goal of agile testing is to deliver early and often
Traditional Testing
Combining source files
Combining software and environment
Combining software and data
Combining software and tests
Combining developers
Agile Testing
Code is frequently checked in
Code is automatically retrieved
Compilation is done automatically
Tests are done automatically
Code reports are generated
Developers get instant feedback
Code is automatically deployed or packaged for delivery
Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.
68
Agile Testing ProcessDevelopers check-in changes as they occurServer detects all changes and initiates testingServer compiles, tests, analyzes, builds, and deploys
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
BuildIntegration
Server
VersionControlServer
BuildScripts
UsesWatches
BuildStatus
ProvidesDeveloper A
Developer B
Developer C
CommitsChanges
CommitsChanges
CommitsChanges
Compile Source Code
Run Unit Tests
Run Coverage TestsStatic Code Analysis
Build Database
Generate Help FilesArchive and Deploy
69
Agile Testing TechnologiesThere are literally hundreds of agile testing toolsThere are tools for building, testing, and deployingIntegration tools monitor repositories and initiate tests
Smart, J. (2009). Automated deployment with maven and friends: Going the whole nine yards. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.
70
Agile Testing StatisticsFewer builds leave in higher bug countsA high number of builds eliminates the defectsGoal is to have as many, early builds as possible
Lacoste, F. J. (2009). Killing the gatekeeper: Introducing a continuous integration system. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 387-392.
71
Scaling Agile TestingAgile testing slows down with very large systemsSlow testing slows integration and increases bugsAgile testing can speed back up with proper attention
Kokko, H. (2009). Increase productivity with large scale CI: Reduce feedback cycle from weeks to 100 minutes. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.
Wide Impact Tuning
Fast builds – less changes – more green
Parallelization of test runs
ClearCase to subversion
Pre-installing as much as possible
Removal of randomness
Compilation in memory
Installation starting parallel with system build
Focused Impact Tuning
More memory and CPUs
Parallelize builds
Replace 3rd party test libraries
Reduce/remove timeouts in tests
Select different tests
Refactor code & components
Tune the network & software
Tune the database
72
Agile Testing CostsMost agile testing tools are “free” open sourceA build server is no more than a commodity PCLow overhead for new and subsequent setup time
Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.
73
Agile Testing BenefitsReduces the cost-of-change by 10 timesFrequent builds dramatically lower defect levelsEnables early “what-if” tests as well as late changes
Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.
when integration/regression testing at each code check-in
Major municipal gas utility
Large retail web site
Global supplier of healthcare equipment
More features and higher quality
Added new functionality 2 weeks before ship
“Oozing Confidence”
74
Agile Testing is NextGenManual testing is CMMI Capability Level 0 or 1Agile testing is a CMMI Capability Level 5 practiceIt is planned, defined, measured, and it’s optimizing
Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.
75
Agile Testing Side-EffectsEliminates big-bang integration in the 11th hourCreates a repeatable and reliable testing processEvaluates system-wide changes throughout project
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
76
AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering Testing
Agile Systems Engineering ValueSummary
77
What is Business Value?Val-ue (văl-'yōō): An amount, quantity, rate, magnitude, or desirability; Economic worth
An economic estimation of the tangible worth of the organizational assets such as buildings and equipmentAn appraisal of intangible assets such as knowledge, experience, skills, patents, processes, and methodsA technique for evaluating the costs and benefits of investments in a business, operations, or personnelThe economic impact of deploying a new product development approach such as systems engineeringThe total life cycle costs of institutionalizing lean and agile systems engineering techniques in an enterprise
78
Agile (138 pt.) and traditional methods (99 pt.)Agile methods fare better in all benefits categoriesAgile methods 359% better than traditional methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Studies of Agile Methods
79
Productivity of Agile MethodsPP productivity 32X more than trad. methodsScrum productivity 5X more than trad. methodsAgile methods productivity 20X more than traditional
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
33.4044
29.28
21.2374
16.1575
5.4436
1.06190
8
16
24
32
40
PP TDD Agile XP Scrum CMMI®
Software Method
Pro
duct
ivit
y (L
OC
/Hou
r)
80
Quality of Agile MethodsXP quality 13X better than trad. methodsScrum quality 3X better than trad. methodsAgile methods quality 5X better than traditional
0.7466
1.7972 2.155 2.355
3.945
9.667
0
2.2
4.4
6.6
8.8
11
XP Agile TDD PP Scrum CMMI®
Software Method
Qua
lity
(Def
ects
/KLO
C)
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
81
Costs of Agile MethodsXP costs 8X less than traditional methodsScrum costs 2X less than traditional methodsAgile methods cost 5X less than traditional methods
$136,551$226,807 $249,653 $265,436
$578,202
$1,108,233
$0
$325,000
$650,000
$975,000
$1,300,000
XP Agile TDD PP Scrum CMMI®
Software Method
Cos
ts
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
82
Benefits of Agile MethodsXP benefits 1.5X more than traditional methodsScrum benefits 1.3X more than traditional methodsAgile methods benefits 1.4X more than trad. methods
$4,372,282$4,282,026 $4,259,180 $4,243,397
$3,930,631
$3,023,064
$2,750,000
$3,237,500
$3,725,000
$4,212,500
$4,700,000
XP Agile TDD PP Scrum CMMI®
Software Method
Ben
efit
s
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
83
ROI of Agile MethodsXP ROI 18X more than traditional methodsScrum ROI 3.4X more than traditional methodsAgile methods ROI 10X more than trad. methods
3,102%
1,788%1,606% 1,499%
580%
173%
0%
925%
1,850%
2,775%
3,700%
XP Agile TDD PP Scrum CMMI®
Software Method
Ret
urn
on In
vest
men
t
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
84
NPV of Agile MethodsXP NPV 2.4X more than traditional methodsScrum NPV 1.9X more than traditional methodsAgile methods NPV 2.3X more than trad. methods
$3,649,388$3,480,979 $3,438,351 $3,408,902
$2,825,313
$1,509,424
$1,000,000
$1,787,500
$2,575,000
$3,362,500
$4,150,000
XP Agile TDD PP Scrum CMMI®
Software Method
Net
Pre
sent
Val
ue
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
85
Real Options of Agile MethodsXP ROA 1.6X more than traditional methodsScrum ROA 1.4X more than traditional methodsAgile methods ROA 1.6X more than trad. methods
$4,265,936$4,105,884 $4,066,678 $4,040,377
$3,608,772
$2,633,052
$2,200,000
$2,825,000
$3,450,000
$4,075,000
$4,700,000
XP Agile TDD PP Scrum CMMI®
Software Method
Rea
l Opt
ions
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
86
AgendaIntroductionSystems EngineeringSystems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering Value
Summary
87
SummaryAgility is the evolution of management thoughtConfluence of traditional and non-traditional ideasImprove performance by over an order-of-magnitude
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing.
Systems engineering approaches
...
New product development approaches
Expertly designed to be fast and efficient
Intentionally lean and free of waste (muda)
Systematic highly-disciplined approaches
Capable of producing high quality systems
Right-sized, just-enough, and just-in-time tools
Intended to maximize business value for customers
88
New Book on Agile MethodsGuide to Agile Methods for business leadersCommunicates business value of Agile MethodsRosetta stone to Agile Methods for traditional folks
http://davidfrico.com/agile-book.htm (Description)http://www.amazon.com/dp/1604270314 (Amazon)
Table of Contents 1. Introduction to Agile Methods 2. Values of Agile Methods 3. History of Agile Methods 4. Antecedents of Agile Methods 5. Types of Agile Methods 6. Practices of Agile Methods 7. Agile Project Management 8. Agile Software Engineering 9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Surveys of Agile Methods14. Costs-Benefits of Agile Methods15. ROI Metrics of Agile Methods16. Measures of Agile Methods17. Costs of Agile Methods18. Benefits of Agile Methods19. ROI of Agile Methods20. NPV of Agile Methods21. Real Options of Agile Methods22. Business Value of Agile Methods23. Agile vs. Traditional Methods24. Future of Agile Methods