Post on 26-Aug-2020
transcript
January 16, 2017 Sam Siewert
SE420 Software Quality Assurance
Lecture 1 – Introduction Part-2
Course Learning Objectives Theory of Overall SQA Process Process Models (Waterfall, Spiral, XP) using Agile Strategy Terminology (IEEE SWEBOK – Chapter 4), Prepare for ISTQB Foundation Certification Principles and Methods for SQA Organization for SQA Practice – Design Methods, Software Construction and Testing (Exam-1)
1. Requirements and Use of Basic Design Models (SA/SD from SE300, OOA/D from SE310) 2. Code Walk-throughs 3. Debugging 4. Feature Addition and Performance Tuning 5. Unit Testing, Exit Criteria (CSUs) 6. Code Inspection 7. Validation and Verification (Code Implements Design, Design is Correct)
Practice, and Scaffold for Final – Requirements and Design (Exam-2) 1. Integration and Test (CSUs to Build and Test CSCI) 2. System Testing (CSC) 3. Acceptance Testing 4. Regression and Test Automation 5. Overall Metrics, Process Improvement (SEI CMM) 6. Delivery and Release Notes 7. SQA Inspection
Sam Siewert 2
January 12, 2016 Sam Siewert
Linux Skills
Introduction Session Part-2
Sam Siewert 4
C Code Unit Development and Test Assignment #1 Questions? Examples-Crypto.zip Hands on Sessions to:
1. Download and unzip code 2. Build code 3. Modify Makefiles (for debug with –g) 4. Re-build 5. Debug with DDD 6. Inspect variable values during debug 7. Set breakpoints 8. Step Over 9. Step Into 10. … 11. If code SEGFAULTS, dump core and
debug the core
Powerful Features of Visual Debug Code Walk-throughs – Static Analysis – Dynamic too
Display Data Structures – Evolution – Recursive – Link List – Trees – Graphs – Multiple
Indirections
Sam Siewert 5
CS317 B-tree Visualized
Discussion… SQA History (read SQA Text Chapter #1) – Come to Class Prepared to Discuss – Gurus (e.g. Edward Deming, Grady Booch, Barry Boehm,
Edward Yourdon, Tom DeMarco, …) – Organizations for SQA - http://www.sei.cmu.edu/ – Process and Validation of Process -
http://whatis.cmmiinstitute.com/
Assignment #1 Discussion – I will Post Every Other Tuesday, We’ll Discuss, Due Following
Week on Friday – Late Assignments – 10% Penalty for 3-day late Turn-in [Sunday],
After Monday, only with Instructor Permission
Sam Siewert 6
Wisdom from QA, SQA & SWE Gurus W. Edwards Deming – “"You can expect what you inspect."
Edsger Dijkstra - “program testing can be used to show the presence of bugs, but never to show their absence” – E.W. Dijkstra, “Notes on Structured Programming,” T.H.-Report
70-WSE-03, Technological University, Eindhoven, 1970; http://www.cs.utexas.edu/users/EWD/ewd02xx/EWD249.PDF
Barry Boehm – Are we building the right product? [validation]; Are we building the product right? [verification]
Sam Siewert 7
Overall SQA Process IEEE Computer SWEBOK version 3 – Organizations for SQA - http://www.sei.cmu.edu/ – Process and Validation of Process -
http://whatis.cmmiinstitute.com/
Process Must Be Defined, Repeatable, and Improved Over Time – Cost of Mistakes in Early Phases is Higher (E.g. Requirements
or Design mistakes compared to Implementation) – Organizational and Individual Engineer SQA – More than One Acceptable Process (E.g. Agile strategy hosting
Spiral process, using OO & Structured analysis and design)
Phases Do Not Have to Be Fully Completed? Waterfall?
Sam Siewert 8
Dimensions of SW Quality 1. Specification (What) – What will be built 2. Design (How) – How it will be Built (interface, function,
protocol) 3. Development (How) – Software Construction 4. Conformance – Was the right thing built (validation) and does
it work (verification)
Basic Interrogatives (Start of Any New Project) – Who? – Team – Where? – Organization (Distributed Team?) – Why? – Marketing Study, Product Research, Customer Request – What? – Capabilities and Requirements Specification – When? – Schedules
Enemy of Quality? Estimation with Defined Process? When is Testing Done? (Exit Criteria)
– How? – Design Details and Software Construction
Sam Siewert 9
Dimensions form Outline for SQA Process
Concept, Statement of Work, Marketing Study, Team Building 1. SW Requirements
– Analysis, Specification, Validation 2. SW Design
– High Level – Interfaces, Data flow, Concurrency, Objects/Modules – Detailed – State Machines, Features and Function, Algorithms – Design Validation (E.g. Simulation with MATLAB)
3. SW Construction 4. SW Testing
– Unit Testing – Integrated Testing – Regression Testing – Acceptance Testing
5. Maintenance – Field Support, Sustaining Engineering Sam Siewert 10
Waterfall Model Complete One Phase and Do Not Return Waterfall with Feedback Modification (Go Back One or More Phases) Defined and Disciplined, but Criticized as Impractical – Too Much Up Front Effort – Difficult to Collaborate – Rigid
Sam Siewert 11
Requirements
Design
Construction
Testing
Maintenance
Evolutionary or Spiral Model Phases, But Radial Distance and Area of Phase Represents Effort (Time / Cost) Plan to Re-visit Phases Goal is to Control Cost/Effort and Impact of Mistakes
Sam Siewert 12
Project Proposal
Reference Project
Specification (Analysis, requirements, Validation)
Design (High Level, Detailed, Validation)
Software Construction (Interfaces, Objects/Modules, Algorithms, Coding)
Compliance (Testing: Unit, Integrated, Regression, Acceptance)
Project HLD
Evolutionary Prototype
I & T Readiness
Detailed, Validated Specification
Detailed Design
All Modules Complete & Unit Tested
System Demonstration (Acceptance) Write/review
Define Services And interfaces
Concept Code (walk-through)
Unit Testing
Refine specification and re-validate Finalize Design
Finalize Code Modules and Unit Tests
(Code inspections)
System Integration Test
SQA – Many Activities to Coordinate SQA Management is Not Simple (Parallel with SWE Development) Activities Need to Be Scheduled at Appropriate Phase – Can’t have Code Inspection at a Walk-through – Can’t Do Integrated Testing Prior to Unit Tests
Defined, Repeatable, Measureable, Repeatable – Basic Requirements for Process – May Tailor for Market
Mission Critical Avionics Software In-Flight Infotainment System Mobile App for Android Storage Data Path Digital Media System
Sam Siewert 13
Stretch Goals and Objectives Focus on Applying Requirements and Design Methods for SQA Use Case Studies (SQA Pitfalls) What Can Go Wrong? Why it Did? Cost to Fix? Requirements V&V (System and Acceptance Test) Design V&V (Unit and I&T) Agile Strategy for Evolutionary Process in SQA
Sam Siewert 14
SQA Knowledge Survey Unit test drivers should attempt to modify the code being tested as little as possible: A_TRUE B_FALSE D_DON’T KNOW Edsger Dijkstra observed that “Program testing can be used to show the presence of bugs, but never to show their absence!”, so we most often define exit criteria for testing modules in terms of metrics such as path coverage rather than a zero defect claim: A_TRUE B_FALSE D_DON’T KNOW Path coverage criteria can be used only for White-box testing: A_TRUE B_FALSE D_DON’T KNOW Sam Siewert 15
SQA Knowledge Survey Black-box testing requires a mixed-mode source/assembly debugger, disassembly of source code into instructions and careful attention to compiler features such as short-circuit logic and conditional execution of instructions: A_TRUE B_FALSE D_DON’T KNOW Regression testing includes tests from all other phases of test development with the purpose to make sure that when a defect is fixed, that somehow new bugs are not introduced : A_TRUE B_FALSE D_DON’T KNOW Integration testing focuses on testing module-to-module interfaces and communication protocols to verify and validate module interfaces before they are integrated to compose an architecture design: A_TRUE B_FALSE D_DON’T KNOW Sam Siewert 16
SQA Knowledge Survey Module designs and code should be tested only for correct function and not stress, performance, or soak tested: A_TRUE B_FALSE D_DON’T KNOW Stress and soak time testing of a module with a unit test driver might reveal a memory leak in the module: A_TRUE B_FALSE D_DON’T KNOW Regression testing should only be run before a software product is shipped: A_TRUE B_FALSE D_DON’T KNOW If SQA Design is Concurrent with Software Design, Acceptance Testing would best be developed during development of: A_REQTS B_DESIGN C_CODING D_DON’T KNOW
Sam Siewert 17
SQA Knowledge Survey Ideally an acceptance test should be provided by the customer, but this rarely happens, so the software engineering team often must help define the acceptance test: A_TRUE B_FALSE D_DON’T KNOW System testing involves configuration of sub-system(s) from multiple software modules and a driver or external stimulus to exercise this sub-system: A_TRUE B_FALSE D_DON’T KNOW System test involves integration of sub-systems into a system tested in an end-to-end test environment, similar to how the system will be used: A_TRUE B_FALSE D_DON’T KNOW Sam Siewert 18