21 1
Continuing Best PracticesContinuing Best Practices
►Slide information taken in large part Slide information taken in large part from former Rational Corporation from former Rational Corporation slides. Considerably modified and slides. Considerably modified and supplemented for classroom use.supplemented for classroom use.
21 2
Practice 5: Verify Software QualityPractice 5: Verify Software Quality
Control Changes
Develop Iteratively
Use Component
Architectures
Manage Requirements
Model Visually VerifyVerify
QualityQuality
Quality, as used within Unified Process, is defined as “The characteristic of having demonstrated the achievement of producing a product which meets or exceeds agreed upon requirements as measured by an agreed upon measures and criteria And is produced by an agreed upon process.
If done this way, the process can be repeated and managed
In most organizations, testing accounts for 30-50% of development costs! Yet most people believe software is not adequately tested when delivered. Testing is difficult; complete testing is impossible; a good process and automated tools help!
21 3
Software problems are 100 to 1000 times more costly to find and repair after deployment
Development Deployment
Cost
Practice 5: Verify Software QualityPractice 5: Verify Software Quality
One of the ways to improve quality: Test early and continuously! Test functionality, reliability; performance; Test architectural decisions early.
21 4
Iterative Development Permits Iterative Development Permits ContinuousContinuous TestingTesting – ensuring higher – ensuring higher QualityQuality
T I M ETest Test Test
TC
DR
Iteration 1 Iteration 2 Iteration 3
TC
DR
TC
DR
Test Life
CycleAssess
PlanDesign
ImplementExecute
Assess!
PlanDesign
ImplementExecute
Assess!
PlanDesign
ImplementExecute
21 5
Verifying Quality – and Continuous Verifying Quality – and Continuous Testing (continued)Testing (continued)
► Rather than test one time, spread testing out continuously. Part of each iteration – BUT (see below) Part of each iteration – BUT (see below)
► Each iteration produces an executable release (not necessarily a product release…) Don’t think of an ‘executable’ as just an .exe or .dll. The Don’t think of an ‘executable’ as just an .exe or .dll. The
executables may be part of an architecture……executables may be part of an architecture…… Each iteration is tested and integrated into an evolving application.
► Note: In the RUP, Each ‘phase’ has one or more iterations, and each phase has milestones!
► Be careful: The ‘extent (how much)’ of R, D, C, T depends on which phase the iteration is in!
► (See your drawings of the UP)See your drawings of the UP)
21 6
Testing in an Iterative EnvironmentTesting in an Iterative Environment R
eq
uir
em
en
tsR
eq
uir
em
en
ts
Test Suite 1Test Suite 1
Iteration 2Iteration 2 Iteration 3Iteration 3 Iteration 4Iteration 4
Test Suite 2Test Suite 2 Test Suite 3Test Suite 3 Test Suite 4Test Suite 4
Iteration 1Iteration 1
Au
tom
ate
d T
est
sA
uto
mate
d T
est
s
Continuous integration!!! (one of the major problems of SDLC!)We will produce automated tests (IF AVAILABLE in your ‘environment’)As new requirements are added in iterations, new tests are generated and run.This means that some tests will be rerun – part of ‘Regression Testing.’
21 7
Quality – What is it?Quality – What is it?
► It is an elusive termIt is an elusive term►Means different things to different Means different things to different
peoplepeople
Next few slides from OOSE text slides; modified
21 8
Software Quality...Software Quality...►Usability - Users can learn it fast and get theirUsability - Users can learn it fast and get their
job done easily job done easily (usually addressed in Interface)(usually addressed in Interface)
►Efficiency - It doesn’t waste resources like CPUEfficiency - It doesn’t waste resources like CPU time and memory time and memory (addressed in execution)(addressed in execution)
►Reliability - It does what it is required to doReliability - It does what it is required to do
without failing (MTBF, MTTR…)without failing (MTBF, MTTR…)►Maintainability - It can be easily changedMaintainability - It can be easily changed►Reusability - Its parts can be used in otherReusability - Its parts can be used in other
projects, so reprogramming is not neededprojects, so reprogramming is not needed
21 9
Software Quality...again, means Software Quality...again, means differentdifferent things to things to differentdifferent
stakeholdersstakeholders
QUALITY SOFTWARE
Developer: easy to design; easy to maintain; easy to reuse its parts
User: easy to learn; efficient to use; helps get work done
Customer: solves problems at an acceptable cost in terms of money paid and resources used
Development manager: sells more and pleases customers while costing less to develop and maintain
21 10
Software QualitySoftware Quality►The different qualities can The different qualities can conflictconflict.. Increasing efficiency can reduce maintainability or Increasing efficiency can reduce maintainability or reusabilityreusability
Increasing usability can reduce efficiencyIncreasing usability can reduce efficiency Increasing usability can reduce maintainabilityIncreasing usability can reduce maintainability Increasing maintainability can reduce efficiency, etc!Increasing maintainability can reduce efficiency, etc!
►Setting Setting objectives for qualityobjectives for quality is a key is a key engineering activityengineering activity Then Then designdesign to meet these objectives to meet these objectives Avoids ‘over-engineering’ which wastes moneyAvoids ‘over-engineering’ which wastes money Obtain the highest possible reliability using a fixed budgetObtain the highest possible reliability using a fixed budget
21 11
Short Term Vs. Long Term Short Term Vs. Long Term QualityQuality
►Short term:Short term: Does the software meet the customer’s Does the software meet the customer’s immediateimmediate
needs? needs? Is it sufficiently efficient for the volume of data we Is it sufficiently efficient for the volume of data we
have have todaytoday??
►Long term: (more in terms of ‘quality factors’Long term: (more in terms of ‘quality factors’ MaintainabilityMaintainability Customer’s future needs Customer’s future needs ScalabilityScalability
21 12
Dimensions of Software QualityDimensions of Software Quality
Functionality
Reliability
Application Performance
System Performance
Does my app do what’s required?
Does my app leak memory?
Does my app respond acceptably?
Does my system perform under production load?
Test cases for each scenario implemented
Analysis tools and code instrumentation
Check performance for each use-case/scenario implemented
Test performance of all use-cases under authentic and worst-case load
Type Why? How?
For each iteration, do the ‘above’ software quality checks.Remember: tests are ‘driven’ by Use Cases and Supplementary Specifications!
21 13
Problems Addressed by Verifying Problems Addressed by Verifying QualityQuality
Testing provides Testing provides objectiveobjective project status project status assessmentassessment
Objective assessment Objective assessment exposes inconsistencies exposes inconsistencies early (early (continuous continuous integration helps!)integration helps!)
Testing and verification are Testing and verification are focused on high risk areasfocused on high risk areas
Defects are found earlier Defects are found earlier and are less expensive to and are less expensive to fix (because ‘testing’ is fix (because ‘testing’ is distributed…distributed…
Automated testing tools Automated testing tools provide testing for provide testing for reliability, functionality, reliability, functionality, and performanceand performance
Root CausesRoot Causes SolutionsSolutions Insufficient requirementsInsufficient requirements Ambiguous Ambiguous
communicationscommunications Brittle architecturesBrittle architectures Overwhelming Overwhelming
complexitycomplexity Subjective Subjective
assessment assessment Undetected Undetected
inconsistenciesinconsistencies
Poor testingPoor testing Waterfall developmentWaterfall development Uncontrolled changeUncontrolled change
Insufficient Insufficient automationautomation
21 14
Practice 6: Practice 6: ControlControl ChangesChanges to to SoftwareSoftware
Control ChangesControl Changes
Develop Iteratively
Use Component
Architectures
Manage Requirements
Model Visually
VerifyQuality
Must recognize that we CANNOT STOP CHANGE. Our goal is to Manage Change!The only ‘constant’ is ‘change!’Must control How and When control is introduced and who introduces the changes. This DOES NOT MEAN that we absolutely accept ALL changes, But…(Discuss!)Want a process that can respond to change…(UP)Must synchronize Change across development teams and locations too. (What impacts do proposed changes have on our architecture!)
21 15
Without explicit control, parallel development degrades to chaos!!!!
Practice 6: Control Changes to Practice 6: Control Changes to SoftwareSoftware
► Consider: we often have:Consider: we often have: Multiple developersMultiple developers Multiple teamsMultiple teams Multiple sitesMultiple sites Multiple iterationsMultiple iterations Multiple releasesMultiple releases Multiple projectsMultiple projects Multiple platformsMultiple platforms
May have multiple developers organized into different teams at multiple sites all working together on multiple iterations, releases, products, and platforms (mostly based on the software architecture)
All at the same time!
21 18
Problems Addressed by Controlling Problems Addressed by Controlling ChangesChanges
Requirements change; workflow Requirements change; workflow is defined and repeatableis defined and repeatable
Change requests facilitate clear Change requests facilitate clear communicationscommunications
Isolated workspaces reduce Isolated workspaces reduce interference from parallel workinterference from parallel work
Change rate statistics are good Change rate statistics are good metrics for objectively assessing metrics for objectively assessing project statusproject status
Workspaces contain all artifacts, Workspaces contain all artifacts, facilitating consistencyfacilitating consistency
Change propagation is Change propagation is controlledcontrolled
Changes maintained in a robust, Changes maintained in a robust, customizable systemcustomizable system
Root CausesRoot Causes SolutionsSolutions Insufficient Insufficient
requirementsrequirements Ambiguous Ambiguous
communicationscommunications Brittle architecturesBrittle architectures Overwhelming Overwhelming
complexitycomplexity Subjective Subjective
assessment assessment Undetected Undetected
inconsistenciesinconsistencies Poor testingPoor testing Waterfall developmentWaterfall development Uncontrolled changeUncontrolled change Insufficient Insufficient
automationautomation
21 19
Best Practices Best Practices ReinforceReinforce Each Each OtherOther
Control Control ChangesChanges
Develop Develop IterativelyIteratively
Use Use ComponentComponent
ArchitecturesArchitectures
Model Model VisuallyVisually
VerifyVerifyQualityQuality
Ensure users involved as requirements evolve
Validate architecturedecisions early on. Drives development,planning, change control. ….
Addresse complexity of design/implementation incrementally.Need tools/support environment!
Measure quality early and often.Continuous testing and integration
Evolve baseline incrementallyArchitecture teams localizing changes; Need CMS, Config Control…
Manage Manage RequirementsRequirements
Remember: best practices yield best results WHEN USED COLLECTIVELY!
What does iterative development do??
21 20
Summary: Best Practices of Software Summary: Best Practices of Software EngineeringEngineering
The result is software that isThe result is software that is On TimeOn Time On Budget On Budget Meets/Exceeds Users NeedsMeets/Exceeds Users Needs
ProjectManager
PerformanceEngineer
ReleaseEngineer
Analyst
Developer
Tester
Control Control ChangesChanges
Develop IterativelyDevelop Iteratively
Use Use ComponentComponent
ArchitecturesArchitectures
Manage Manage RequirementsRequirements
Model Model VisuallyVisually VerifyVerify
QualityQuality
