0 Copyright 2006 Susan K. (Kathy) Land Best Practices for Software and Systems Engineering Documentation Using IEEE Software and System Engineering Standards to Support America’s Army Software and Systems Technology Conference (SSTC) 2006 Susan K. (Kathy) Land, CSDP IEEE 2 nd Vice President, Standards Activities Production Planner America’s Army Northrop Grumman Corporation [email protected]MAJ Bret Wilson Chief Technology Officer America’s Army Office of Economic Manpower and Analysis (OEMA) [email protected]
Transcript
0Copyright 2006 Susan K. (Kathy) Land
Best Practices for Software and Systems Engineering Documentation
Using IEEE Software and SystemEngineering Standards to Support America’s Army
Software and Systems Technology Conference (SSTC) 2006
Susan K. (Kathy) Land, CSDPIEEE 2nd Vice President, Standards ActivitiesProduction Planner America’s ArmyNorthrop Grumman [email protected]
MAJ Bret WilsonChief Technology Officer America’s ArmyOffice of Economic Manpower and Analysis (OEMA)[email protected]
Why IEEE Standards?Why IEEE Standards?Why Process Improvement?Why Process Improvement?Overview of the CMMIOverview of the CMMICMMI Implementation CMMI Implementation –– IDEALIDEALCMMI CMMI –– Direct IEEE Standards SupportDirect IEEE Standards SupportImplementation SummaryImplementation SummaryCommon PitfallsCommon PitfallsCurrent Application to America’s ArmyCurrent Application to America’s Army
2Copyright 2006 Susan K. (Kathy) Land
IEEE Software & Systems Engineering Standards:IEEE Software & Systems Engineering Standards:
Define software and systems Define software and systems engineering (SE) processes.engineering (SE) processes.Ensure CMMIEnsure CMMI--SW Level 2 compliance.SW Level 2 compliance.Perform software engineeringPerform software engineering
gap analyses.gap analyses.Improve existing SE processes.Improve existing SE processes.Eliminate the chaos…Eliminate the chaos…
3Copyright 2006 Susan K. (Kathy) Land
Why IEEE Standards?Why IEEE Standards?IEEE Standards can be used as IEEE Standards can be used as toolstools to help in the to help in the painful process ofpainful process of ‘‘selfself--documentation’documentation’..
Many of the Many of the standards provide detailed procedure standards provide detailed procedure explanationsexplanations, they offer section by section guidance on , they offer section by section guidance on building the necessary documentation.building the necessary documentation.
Most importantly, they Most importantly, they provide the best practiceprovide the best practice as as defined by those from the software development defined by those from the software development industry who sit on the panels of reviewers.industry who sit on the panels of reviewers.
4Copyright 2006 Susan K. (Kathy) Land
Why Process Improvement?Why Process Improvement?All those practicing as software engineers should desire All those practicing as software engineers should desire to evolve out of the chaotic activities and heroic efforts to evolve out of the chaotic activities and heroic efforts of a Level 1 organization. of a Level 1 organization.
Because no one likes a ‘painful’ work environment Because no one likes a ‘painful’ work environment --
Good software can be developed by a Level 1 Good software can be developed by a Level 1 organization, but often at the expense of the developers. organization, but often at the expense of the developers.
People get tired of being the hero People get tired of being the hero --
At the repeatable level, Level 2, software engineering At the repeatable level, Level 2, software engineering processes are under basic management control and processes are under basic management control and there is a management discipline.there is a management discipline.
Even the most dieEven the most die--hard techie needs time away from work hard techie needs time away from work --
5Copyright 2006 Susan K. (Kathy) Land
The CMMI and IEEE SWE StandardsThe CMMI and IEEE SWE Standards
The The CMMICMMI is a compendium of software is a compendium of software engineering practices, which act as engineering practices, which act as
the motivator for the continuous evolutionthe motivator for the continuous evolutionof improved software engineering processes. of improved software engineering processes.
IEEE StandardsIEEE Standards can be used to can be used to provide the basic beginning frameworkprovide the basic beginning framework
for software process improvement.for software process improvement.
CMMICMMI--SW Level 2 / The SpecificsSW Level 2 / The SpecificsRequirements Management. Manage requirements associated with a project and identify inconsistencies between the requirements and the project plan and associated work products.
Project Planning. Planning in support of project activities.
Project Monitoring and Control. Effective management of a software project.
Process and Product Quality Assurance. Activities associated with software project oversight.
Configuration Management. Definition, control, review, and reporting of the work products associated with a software project.
Supplier Agreement Management. Acquisition of products from suppliers for which there exists a formal agreement.
Measurement and Analysis. Development, maintenance, and implementation of software project measurement activities.
8Copyright 2006 Susan K. (Kathy) Land
CMMICMMI--SW CrossSW Cross--ReferenceReference
Requirements Management IEEE Std 830 – 1998IEEE Recommended Practice for Software Requirements Specifications
Project Planning IEEE Std 1058 – 1998 IEEE Standard for Software Project Management Plans; 1490 - PMBOK
Project Monitoring and Control IEEE Std 1058 – 1998IEEE Standard for Software Project Management Plans
Process and Product Quality Assurance
IEEE Std 730 – 2002IEEE Standard for Software Quality Assurance
Configuration Management IEEE Std 828 – 1998IEEE Standard for Software Configuration Management Plans
Supplier Agreement Management IEEE Std 1062 – 1998IEEE Recommended Practice for Software Acquisition
Measurement and Analysis IEEE Std 1045 – 2002IEEE Standard for Software Productivity Metrics
Level 2 CMMI-SW PAIEEE Standard(s)
9Copyright 2006 Susan K. (Kathy) Land
11stst -- Framework DefinitionFramework DefinitionSoftware Life CycleSoftware Life Cycle
IEEE/EIA 12207.0IEEE/EIA 12207.0, , Industry Implementation of International Industry Implementation of International Standard ISO/IEC12207:1995 Standard ISO/IEC12207:1995 ——Standard for Information Standard for Information Technology Technology ——Software life cycle processesSoftware life cycle processes
IEEE/EIA 12207.1IEEE/EIA 12207.1, , Industry Implementation of International Standard Industry Implementation of International Standard ISO/IEC12207:1995 ISO/IEC12207:1995 —— (ISO/IEC 12207) Standard for Information (ISO/IEC 12207) Standard for Information Technology Technology ——Software life cycle processes Software life cycle processes –– Life Cycle DataLife Cycle DataIEEE/EIA 12207.2IEEE/EIA 12207.2, , Industry Implementation of International Standard Industry Implementation of International Standard ISO/IEC12207:1995 ISO/IEC12207:1995 —— (ISO/IEC 12207) Standard for Information (ISO/IEC 12207) Standard for Information Technology Technology ——Software life cycle processes Software life cycle processes –– Implementation considerationsImplementation considerations
Systems Life CycleSystems Life CycleISO/IEC 15288ISO/IEC 15288, , Systems engineering Systems engineering ——
System life cycle processesSystem life cycle processes
•Developed to support the CMM/CMMI• Serves a road map to software process implementation and improvement
12Copyright 2006 Susan K. (Kathy) Land
Define and Train the Process Team Define and Train the Process Team (Initiate)(Initiate)
Identify a group of people who are given responsibility and authority for improving organizational processes:
• Implementing process improvement can be very time-consuming, depending upon the scope and complexity of the effort. • Expectations for each team member’s time commitments and job responsibilities must be modified accordingly to reflect the new responsibilities. • This commitment should reflect time budgeted for process definition and improvement and any required refresher training.
IEEE software engineering standards provide valuable support to the process team. The standards should be used to help define and document the initial baseline of recommended processes and practices.
13Copyright 2006 Susan K. (Kathy) Land
Set Realistic Goals (Diagnose)Set Realistic Goals (Diagnose)
The leap from chaos (Level 1) to Level 2 is often the The leap from chaos (Level 1) to Level 2 is often the hardest stephardest step for for many organizations. many organizations.
Defining the Defining the initial process baselineinitial process baseline is key, in order to understand is key, in order to understand where the organization needs to be; it must first understand whewhere the organization needs to be; it must first understand where it is. re it is.
Use the CMMI®Use the CMMI®--SW Level 2 and Level 3 goals to identify areas of SW Level 2 and Level 3 goals to identify areas of weakness or bottlenecks in existing processes. Then refer to eaweakness or bottlenecks in existing processes. Then refer to each of ch of the appropriate IEEE Software Engineering standards using them athe appropriate IEEE Software Engineering standards using them as s planning toolsplanning tools and as and as checklistschecklists to be considered when determining to be considered when determining how to accomplish process completeness.how to accomplish process completeness.
It is important to identify which organizational process plans wIt is important to identify which organizational process plans will be ill be developed and the sequence of their development. developed and the sequence of their development.
14Copyright 2006 Susan K. (Kathy) Land
Fix Timelines (Establish)Fix Timelines (Establish)Goal driven process improvement is the most effective. IdentifyGoal driven process improvement is the most effective. Identify
short and long term goals and time periods; associate these short and long term goals and time periods; associate these goals as schedule milestones.goals as schedule milestones.
Identify responsible individuals.Identify responsible individuals.Identify participating project managers.Identify participating project managers.Identify candidate projects.Identify candidate projects.Solidify backing of Senior Management.Solidify backing of Senior Management.Look at existing processes.Look at existing processes.Define the formats for your process plans using IEEE Define the formats for your process plans using IEEE Software Engineering Standards and measure them against the Software Engineering Standards and measure them against the CMMI® requirements.CMMI® requirements.Get project members to provide feedback on process plans, Get project members to provide feedback on process plans, review and incorporate feedback.review and incorporate feedback.Conduct ARC Class C Gap Analysis.Conduct ARC Class C Gap Analysis.
15Copyright 2006 Susan K. (Kathy) Land
Baseline and Implement Processes (Act)Baseline and Implement Processes (Act)
Use IEEE standards to develop your baseline process Use IEEE standards to develop your baseline process documentation. documentation.
Once a process baseline has been established formulate an Once a process baseline has been established formulate an action plan.action plan.
It is also important to evaluate and identify any potential toolIt is also important to evaluate and identify any potential tools s that may be used in support of process automation:that may be used in support of process automation:
A tool is not a substitute for a process.A tool is not a substitute for a process.An ideal candidate area for this type of automation is SCM.An ideal candidate area for this type of automation is SCM.
Many IEEE SWE standards provide documentation templates Many IEEE SWE standards provide documentation templates and describe in detail what the processes should contain. and describe in detail what the processes should contain.
Think of these standards as an Think of these standards as an inin--house software process consultanthouse software process consultantwho has recommended, based upon who has recommended, based upon years of experienceyears of experience, the , the proper methodologies and techniques to be used in support of proper methodologies and techniques to be used in support of software development.software development.
16Copyright 2006 Susan K. (Kathy) Land
Perform Gap Analysis (Learn)Perform Gap Analysis (Learn)
It is important to gauge how effectively process It is important to gauge how effectively process improvements have been implemented for continuous improvements have been implemented for continuous process improvement to be successful.process improvement to be successful.
Develop a benchmarking appraisal to support gap analysis Develop a benchmarking appraisal to support gap analysis activities. activities. Provides a baseline for future process improvement efforts Provides a baseline for future process improvement efforts and will identify weaknesses and strengths. and will identify weaknesses and strengths. Review the associated appraisal methodology used in support Review the associated appraisal methodology used in support of the CMMI®of the CMMI®
Common Implementation PitfallsCommon Implementation Pitfalls
Being overly prescriptiveBeing overly prescriptive
Remaining confined to a specific Remaining confined to a specific stagestage
Lack of incentivesLack of incentives
No metrics takenNo metrics taken
19Copyright 2006 Susan K. (Kathy) Land
America’s Army America’s Army –– Project BackgroundProject Background
Project of Economic and Manpower Project of Economic and Manpower Analysis (OEMA) / Col Casey Analysis (OEMA) / Col Casey WardynskiWardynski
Chris Chambers Chris Chambers –– Deputy DirectorDeputy DirectorMAJ Bret Wilson MAJ Bret Wilson –– Creative DirectorCreative Director
Factoids:Factoids:PC game PC game –– 6 Million registered users6 Million registered users29% of U.S. young people recognize the AA brand29% of U.S. young people recognize the AA brandJoint venture with Joint venture with UbisoftUbisoft –– Xbox and PS2 versionsXbox and PS2 versionsCell phone game / Real Hero ProgramCell phone game / Real Hero ProgramHitting the NASCAR circuitHitting the NASCAR circuitOver 30 Government training applicationsOver 30 Government training applications5 PC versions in the pipeline5 PC versions in the pipeline
20Copyright 2006 Susan K. (Kathy) Land
Implementing process with AAImplementing process with AA
Previous operation without centralized Previous operation without centralized software engineering management and software engineering management and oversightoversightGeographically diverse development partnersGeographically diverse development partnersNon traditional software developmentNon traditional software developmentDiverse applications and release schedulesDiverse applications and release schedulesProduct integration challengesProduct integration challengesQA and testing challengesQA and testing challengesCM challengesCM challenges
21Copyright 2006 Susan K. (Kathy) Land
Formation of SED Project OfficeFormation of SED Project Office
CMMI Center of ExcellenceCMMI Center of ExcellenceDemonstration LabDemonstration Lab
Signed into official PO status 12/2005Signed into official PO status 12/2005
22Copyright 2006 Susan K. (Kathy) Land
Geographically Diverse DevelopmentGeographically Diverse Development
Development Partners Development Partners –– DoDDoDUKUKGermanyGermanyPicatinnyPicatinnyRedstone ArsenalRedstone Arsenal
CommercialCommercialPublic Applications (Emeryville, CA)Public Applications (Emeryville, CA)Zombie (Seattle, WA )Zombie (Seattle, WA )Virtual Heroes (Cary NC)Virtual Heroes (Cary NC)Riptide Software (Los Angeles, CA)Riptide Software (Los Angeles, CA)MoCapMoCap Studio (Monterey Bay, CA)Studio (Monterey Bay, CA)……
23Copyright 2006 Susan K. (Kathy) Land
NonNon--Traditional Software DevelopmentTraditional Software Development
Gaming is known for being ‘Extreme’Gaming is known for being ‘Extreme’Many cases this is a synonym for Many cases this is a synonym for AdhocAdhocToo many islandsToo many islands
Development partnersDevelopment partnersSeparate companiesSeparate companiesDoDDoD and commercial mixand commercial mixNo centralized management No centralized management –– some hostilitysome hostility
UnRealUnReal Scripting and the job marketScripting and the job marketLicense from Epic softwareLicense from Epic softwareSmall resource poolSmall resource poolModeling and simulation expertiseModeling and simulation expertise
AAPA is the product integrator AAPA is the product integrator –– Public PC GamePublic PC GameReceipt of products not fully testedReceipt of products not fully testedFinal design authorityFinal design authority
Review of design happening during integration Review of design happening during integration phasephase
The most expensive portion of the lifecycle in terms of The most expensive portion of the lifecycle in terms of cost and schedulecost and scheduleThe most demoralizingThe most demoralizing
Development partners not communicatingDevelopment partners not communicatingNo effective preNo effective pre--integration communicationintegration communication
25Copyright 2006 Susan K. (Kathy) Land
QA and Testing ChallengesQA and Testing Challenges
No insight into Development Partners (DP) No insight into Development Partners (DP) testing:testing:
Not consistent across Not consistent across DPsDPsTest plan development ? Test tools ? Test results ?Test plan development ? Test tools ? Test results ?SED Test ManagerSED Test Manager
AAPA IntegrationAAPA IntegrationLast stop QA/testing prior to public releaseLast stop QA/testing prior to public releaseShifting the responsibility back to the development Shifting the responsibility back to the development teamsteamsPerformancePerformance--Based Statements of WorkBased Statements of Work
26Copyright 2006 Susan K. (Kathy) Land
CM ChallengesCM Challenges
No centralized code repository No centralized code repository Main PC code branch under contractor managementMain PC code branch under contractor management
SED to host Perforce repositorySED to host Perforce repositorySED to host SED to host SharepointSharepoint development portaldevelopment portal
Two distinct code branchesTwo distinct code branchesRelease and future developmentRelease and future developmentRequirement to support production and marketRequirement to support production and marketMerge targeted for 2.7 releaseMerge targeted for 2.7 release
Lack of general CM controls among several Lack of general CM controls among several development teamsdevelopment teams
27Copyright 2006 Susan K. (Kathy) Land
Using IEEE as a GuideUsing IEEE as a GuideInitially take a read… take timeInitially take a read… take timeLean on IEEE Software and Systems Lean on IEEE Software and Systems
Engineering StandardsEngineering StandardsUse the CMMI practicallyUse the CMMI practicallyTwo parts software engineering, one part psychologyTwo parts software engineering, one part psychologyFocus on the software engineering “sequence”Focus on the software engineering “sequence”Keep the documentation flexible, refine the Keep the documentation flexible, refine the requirements as you go along, provide guidance, let requirements as you go along, provide guidance, let the team define their practices. IEEE Standards are the team define their practices. IEEE Standards are an excellent source for this approach. Apply the an excellent source for this approach. Apply the CMMI practically.CMMI practically.Be the most careful with those who need process the Be the most careful with those who need process the most.most.
28Copyright 2006 Susan K. (Kathy) Land
AA Where we are…AA Where we are…
Working integration, QA, testing, and Working integration, QA, testing, and CM challengesCM challengesCompleted Version 2.6 on schedule and budgetCompleted Version 2.6 on schedule and budgetDeveloping 2.7, Developing 2.7, Forecasting 2.8, 2.9, 3.0Forecasting 2.8, 2.9, 3.0Working schedules for 30+ active Working schedules for 30+ active DoDDoD training training applicationsapplications6 active commercial joint partnerships6 active commercial joint partnershipsUnreal 2.5 to Unreal 3.0 migrationUnreal 2.5 to Unreal 3.0 migration
29Copyright 2006 Susan K. (Kathy) Land
Diverse Applications and SchedulesDiverse Applications and Schedules
Show and tell Show and tell ––2.7 Initial requirements vs. final2.7 Initial requirements vs. finalScheduleScheduleDesign examplesDesign examplesCM and Central RepositoryCM and Central RepositoryUnit and integration test planningUnit and integration test planningConfigurable Configurable SOWsSOWsDeveloping standard approachDeveloping standard approach
toward AA product integrationtoward AA product integration
30Copyright 2006 Susan K. (Kathy) Land
SummarySummaryLeverage the expertise contained in the IEEE Leverage the expertise contained in the IEEE Software and Systems Engineering Standards.Software and Systems Engineering Standards.
Fix timelines to produce goal driven process Fix timelines to produce goal driven process improvement.improvement.
Define your processes in outline form.Define your processes in outline form.
Perform a gap analysis.Perform a gap analysis.
Use IEEE standards to develop your baseline Use IEEE standards to develop your baseline process documentation.process documentation.
Perform selfPerform self--audit using CMMI audit using CMMI PAsPAs..
Readjust processes/plans based upon audit results.Readjust processes/plans based upon audit results.
Use practical judgment and gauge your development Use practical judgment and gauge your development environmentenvironment
Make a plan. Then follow the plan. - Watts Humphrey
Make a plan. Then follow the plan. - Watts Humphrey
31Copyright 2006 Susan K. (Kathy) Land
Define software engineering (SE) Define software engineering (SE) processesprocessesPerform software engineering gap Perform software engineering gap analysesanalysesImprove existing SE processesImprove existing SE processesExamples of 102 work products required in support Examples of 102 work products required in support of assessment activities of assessment activities An integrated set of 38 deployable document An integrated set of 38 deployable document templates on CDtemplates on CD--ROMROM
Need More Help? Gratuitous Plug…Need More Help? Gratuitous Plug…
Additional Resources:Additional Resources:S. Land, S. Land, Jumpstart CMM/CMMI Software Process Improvement/Jumpstart CMM/CMMI Software Process Improvement/
Using IEEE Software Engineering StandardsUsing IEEE Software Engineering Standards, John Wiley/, John Wiley/IEEE Press, Feb 2005.IEEE Press, Feb 2005.
IEEE Software Engineering Standards Collection, IEEE Software Engineering Standards Collection, Institute of Electrical Institute of Electrical and Electronics Engineers, Inc. New York, NY, and Electronics Engineers, Inc. New York, NY, 2003.2003.
CMMICMMI®® --SE/SW/IPPD/SS, V1.1SE/SW/IPPD/SS, V1.1, Carnegie Mellon University, , Carnegie Mellon University, Software Engineering Institute, Pittsburgh, PA, March 2002.Software Engineering Institute, Pittsburgh, PA, March 2002.
CMMICMMI®® --SE/SW/IPPD/SS, V1.1, SE/SW/IPPD/SS, V1.1, CMMI for Systems Engineering/Software CMMI for Systems Engineering/Software Engineering/Integrated Product and Process Development, and SuppEngineering/Integrated Product and Process Development, and Supplier Sourcing Version 1.1, lier Sourcing Version 1.1, CMMICMMISMSM --SE/SW/IPPD/SS, V1.1, Continuous Representation.SE/SW/IPPD/SS, V1.1, Continuous Representation. CMU/SEICMU/SEI--CMU/SEICMU/SEI--20022002--TRTR--011, ESC011, ESC--TRTR--20022002--011, Carnegie Mellon University, Software Engineering 011, Carnegie Mellon University, Software Engineering Institute, Pittsburgh, PA, March 2002.Institute, Pittsburgh, PA, March 2002.
[[CrollCroll 2003] Third Annual CMMI2003] Third Annual CMMI®® Technology Conference and Users Group, Technology Conference and Users Group, Eight Steps Eight Steps to Success in CMMI to Success in CMMI –– Compliant Process Compliant Process EngneeringEngneering, , November 2003.November 2003.
[Land 2004] 2004 Software and Systems Technology Conference, [Land 2004] 2004 Software and Systems Technology Conference, The Real World The Real World Application of IEEE Software Engineering Standards, Application of IEEE Software Engineering Standards, June 2004.June 2004.
IEEE/EIA Standard 12207.0IEEE/EIA Standard 12207.0--1996, 1996, Industry Implementation of International Standard Industry Implementation of International Standard ISO/IEC12207:1995 ISO/IEC12207:1995 —— (ISO/IEC 12207) Standard for Information Technology (ISO/IEC 12207) Standard for Information Technology ——Software Software life cycle processes,life cycle processes, Institute of Electrical and Electronics Engineers, Inc. New YoInstitute of Electrical and Electronics Engineers, Inc. New York, NY, rk, NY, 1998.1998.
IEEE/EIA Standard 12207.1IEEE/EIA Standard 12207.1--1997, 1997, Industry Implementation of International Standard Industry Implementation of International Standard ISO/IEC12207:1995 ISO/IEC12207:1995 —— (ISO/IEC 12207) Standard for Information Technology (ISO/IEC 12207) Standard for Information Technology ——Software Software life cycle processes life cycle processes –– Life cycle data,Life cycle data, Institute of Electrical and Electronics Engineers, Inc. Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1998.New York, NY, 1998.
