Post on 01-Jan-2016
transcript
© 2014 Noblis, Inc. Noblis proprietary and confidential.
Using Systems Engineering Processes and Methods to Make Procurements Easier and to Fully Meet Public Transportation Agency Expectations
Blake Christie and Ann Diephaus
April 3, 2014
22© 2014 Noblis, Inc. Noblis proprietary and confidential.
The Transportation Challenge
■ More Vehicles
■ More Drivers
■ More Trips
But –
■ Fewer Resources
■ Space Limits on New Road Construction
■ Frequent Local Opposition to Mass Transit Solutions
Means –
We have congestion, long commutes, dangerous roadways, and …
33© 2014 Noblis, Inc. Noblis proprietary and confidential.
So What’s The Solution?
■ More money would help, but that’s not the total answer
■ Technology can help – Intelligent Transportation Systems
■ Intelligent Transportation Systems (ITS) include:• Better information for the surface transportation system manager (e.g., sensors,
cameras, weather stations)
• Better information for the surface transportation system user (e.g., 5-1-1, travel time maps on the Internet)
• Technology that reduces the time spent in travel-related activities, through the use of automation (e.g., toll collection)
44© 2014 Noblis, Inc. Noblis proprietary and confidential.
Technology Solutions Bring Their Own Challenges
■ New technology is generally proprietary – and expensive
■ Competing technologies don’t start out interoperating, complicating the use of similar items in different locations (e.g., Smart Tag vs. E-Pass)
■ Users don’t necessarily have a “vision” for how to integrate all of the necessary systems• In transportation, engineers are primarily Civil Engineers, experienced in
construction, not systems
• Information systems involve software, which has its own known set of challenges
• In the early days of ITS (and to a lesser degree today), there was the “NIH” syndrome
55© 2014 Noblis, Inc. Noblis proprietary and confidential.
U.S. Department of Transportation Role
■ Fostered research and development of:• “Intelligent Highways”
• Intelligent Transportation Systems (successor to “intelligent highways”)
■ Sponsored the development of a National ITS Architecture• Conceptual description of the expected systems and the infrastructure that ties
them together
• Framework for states and other jurisdictions building intelligent transportation systems
• Adding connected vehicles architecture
• Provides the “Big Picture”
■ Recognized the need for standards to support the development of intelligent transportation systems and sponsored their development
66© 2014 Noblis, Inc. Noblis proprietary and confidential.
Issues With ITS Standards Development
■ Early standards weren’t successful• Failed to capture the full scope of the areas they were intended to address
• Were limited by the lack of systems experience by the people developing them
• Were ignored (because of limitations) by the agencies fielding ITS systems
• Were hard to read and ambiguous
■ Revisions to standards weren’t successful• Groups working on the standards lacked sufficient systems expertise
• Users were inadequately involved in modifications, largely because of a lack of systems expertise – but the standards were intended for them!!
77© 2014 Noblis, Inc. Noblis proprietary and confidential.
Systems Engineering – To the Rescue!
■ Proposal made to use a systems engineering process for ITS standards development
■ Why?• Provide a context for the standard – concept of how it would be used in actual
operation of a system• Develop clear-cut requirements, based on user needs, for the interfaces and
devices requiring standardization• Trace the requirements back to user needs, to show users how the standard
evolved• Design standard solutions that addressed requirements• Trace standard solutions back to requirements• Create mechanism for testing products that claimed conformance to the standard
88© 2014 Noblis, Inc. Noblis proprietary and confidential.
Development of the Systems Engineering Profile for ITS Standards
■ Identified the appropriate “ilities”• Usability• Readability• Maintainability• Interoperability• Flexibility
■ Mapped the interface standards development process to system life-cycle stages• Concept of Operations• Requirements• Needs-to-Requirements Matrix• Design• Requirements Traceability Matrix• Verification and Validation
9© 2014 Noblis, Inc. Noblis proprietary and confidential. 9
The “V” Diagram (Development Life-Cycle)
Conceptof Operations
High LevelRequirements
DetailedRequirements
High LevelDesign
DetailedDesign
Implementation
Operations &Maintenance
SystemAcceptance
SubsystemVerification
Integration &Test
Time
Relates to
Relates to
Relates to
Configuration Control Begins With the First
Product and Continues Throughout the Life of
the System
Development Translates Design Into Product
1010© 2014 Noblis, Inc. Noblis proprietary and confidential.
Systems Engineering and Quality
■ Quality Concerns Exist:• Within each stage• At the transitions between stages
■ Within a stage, solutions must validly reflect the intent and content of the product(s) of the previous stage
■ At the transitions, the stakeholders must verify the products that result from a stage• Each product must be complete• Each product must be correct
Conceptof Operations
High LevelRequirements
DetailedRequirements
High LevelDesign
DetailedDesign
Implementation
Operations &Maintenance
SystemAcceptance
SubsystemVerification
Integration &Test
We want to improve the quality of the standards
1111© 2014 Noblis, Inc. Noblis proprietary and confidential.
What Systems Engineering Provides to ITS Standards Development
■ Creates a quality standard
■ Ensures that the standard meets user needs
■ Verifies that the standard is complete
■ Validates that the standard is correct
■ Supports deployments
■ Reduces cost of interface development
1212© 2014 Noblis, Inc. Noblis proprietary and confidential.
Profile for a ConOps for a Standard
■ Define User Needs• Description of what the users want to do• Highest level of “requirement” in the system
■ Establish Operational Policies and Constraints• What policies govern the operation of the system• What constraints does the system have to accommodate
■ Delineate Modes of Operation• Normal mode (rush hour, scheduled events, etc.)• Other modes (accidents, sever weather, tec.)
■ Provide Operational Scenarios (optional) – used to give examples of how the user (or system) may operate with the capability desired
■ Tell a story■ Introduced use of Guides:
• IEEE Std 1362-1998 (ConOps) • INCOSE SYSTEMS ENGINEERING HANDBOOK, version 3.1, section
3.3.2• IEEE Std 1028-2008 (for technical reviews and walkthroughs)
1313© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced the Characteristics of Good Functional Requirements
■ Necessary
■ Concise (minimal, understandable)
■ Attainable (achievable or feasible)
■ Complete (standalone)
■ Consistent
■ Unambiguous
■ Verifiable
Guided by IEEE 1233-1998 & INCOSE SYSTEMS ENGINEERING HANDBOOK version 3.1, Appendix I
Needs
Requirements
Based
Needs to Requirements Matrix
1414© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced the Profile of Design Concepts
■ Based on the requirements
■ Directly traceable to one or more requirements
■ Consistent with the requirements
■ Follow rule of one (a single) definition(How many times should you define an object?)
Requirements
Designs
Driven
Requirements Traceability Matrix (RTM)
1515© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced Basics of Verification and Validation
Verification and Validation are related concepts
■ Verification is “building the product right” -- ensures that all functions implemented in the product have been implemented correctly
■ Validation is “building the right product” – ensures that the desired functions have been implemented in the delivered product
V&V Methodologies
1616© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced What V&V is at Each Stage
Conceptof Operations
High LevelRequirements
DetailedRequirements
High LevelDesign
DetailedDesign
Implementation
Operations &Maintenance
SystemAcceptance
SubsystemVerification
Integration &Test
V&V for ConOps asks, is this a complete set of needs and is each need correctly
described? V&V for requirements asks, do the requirements address all the
needs (completeness) and do respective requirements fulfill the
need (correctness)?
V&V for the design asks, are the requirement(s) all addressed
(completeness) and does the design fulfill each requirement
(correctness)?
1717© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced the Needs-to-Requirements Matrix
Shows which requirements fulfill specific user needs Every Need Must Be Addressed By At Least One Requirement Every Requirement Must Relate to At Least One Need Any Need That is Not Addressed By At Least One Requirement may be
Irrelevant Every Requirement That Does Not Address At Least One Need is
Irrelevant
1818© 2014 Noblis, Inc. Noblis proprietary and confidential.
What is a Needs-to-Requirements Matrix used for?
■ Provides tool for completeness and correctness (V&V)
■ Helps guide procurements
■ Establishes high level picture (how it fits)
■ Maps (references) to details
■ Sets up basis for design
1919© 2014 Noblis, Inc. Noblis proprietary and confidential.
What Does the Needs-to-Requirements matrix Look like in ITS Standards
User Need ID
User Need FR ID Functional
Requirement Conformance Support Additional Specifications
2.5.2.3.3 Define a Message VMS:M Yes / NA
3.5.1.2.3.1 Determine Maximum Number of Pages
M Yes
3.5.1.2.3.2 Determine Maximum Message Length
M Yes
3.5.1.2.3.3 Determine Supported Color Schemes
M Yes
3.5.2.3.2.3Configure Default Flash-On and Flash-Off Times
O Yes / No
The DMS shall support all flash on times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments. The DMS shall support all flash off times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments.
2020© 2014 Noblis, Inc. Noblis proprietary and confidential.
How is the Needs-to-Requirements Matrix Used to make Procurements Easier?
The system owner selects the operational concepts they need The system owner addressed the optional requirements as needed The filled in needs-to-requirements matrix becomes the requirements
selected for the device interface When included “as specified in the standard” in contract language, off-the-
shelf implementations are achievable Basis for all interface testing
“Overall, VTTI feels that the project has been a resounding success.The specification process meets the goals of creating a more user-friendly environment for an agency to develop procurements.”
Ashwin AmannaVirginia Tech Transportation Institute
Deployment and Testing of an Updated Dynamic Message Sign (DMS) Standard, FINAL REPORT, April 24, 2007
2121© 2014 Noblis, Inc. Noblis proprietary and confidential.
What Does the Needs-to-Requirements matrix Look like in ITS Standards
User Need ID
User Need FR ID Functional
Requirement Conformance Support Additional Specifications
2.5.2.3.3 Define a Message VMS:M Yes / NA
3.5.1.2.3.1 Determine Maximum Number of Pages
M Yes
3.5.1.2.3.2 Determine Maximum Message Length
M Yes
3.5.1.2.3.3 Determine Supported Color Schemes
M Yes
3.5.2.3.2.3Configure Default Flash-On and Flash-Off Times
O Yes / No
The DMS shall support all flash on times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments. The DMS shall support all flash off times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments.
2222© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced the Requirements Traceability Matrix (RTM)
■ Tells developers how specifically to meet a given requirement■ Traces from requirements to design (dialog and associated objects)
• Every Requirement Must Be Addressed By At Least One “Thing” (Dialog, Object, Element)
• Every “Thing” Must Relate Back to At Least One Requirement• Any Requirement That is Not Addressed By At Least One “Thing” is Unfulfilled
(Unsatisfied)• Any “Thing” That Does Not Address At Least One Requirement is Unnecessary
2323© 2014 Noblis, Inc. Noblis proprietary and confidential.
What is a RTM Used For?
■ Provides tool for completeness and correctness checks (V&V) of the design
■ Specifies the “how” for development
■ Maps (or references) requirements to design
2424© 2014 Noblis, Inc. Noblis proprietary and confidential.
What does the RTM Look like in ITS Standards?
FR ID FunctionalRequirement
Dialog ID
Object ID
Object
3.5.2.3.2.3 Configure Default Flash-On and Flash-Off Times
G.3
5.5.3 defaultFlashOn
5.5.5 defaultFlashOff
2525© 2014 Noblis, Inc. Noblis proprietary and confidential.
Slide-25
Example Dialog
Center Signal Controller
GET (activeVolumeOccupancyDetectors)
RESPONSE (activeVolumeOccupancyDetectors)
GET (volumeOccupancyTable)
RESPONSE (volumeOccupancyTable)
Determine number of rows
in table
GET (volumeOccupancySequence)
RESPONSE (volumeOccupancySequence)Determine if a new
table is available
Collect volume & occupancy
data
Read Volume/Occupancy Data Dialogue Sequence
Obj
ects
2626© 2014 Noblis, Inc. Noblis proprietary and confidential.
How is the RTM used to make procurements and testing easier?
The RTM specifies how the interface is to precisely behave Think of it as an Interface Control Document (ICD) When included “as specified in the standard” in contract language, off-the-
shelf implementations are achievable Basis for all interface testing
27© 2014 Noblis, Inc. Noblis proprietary and confidential. 27
Introduction of Advanced Verification Concepts
Conceptof Operations
High LevelRequirements
DetailedRequirements
High LevelDesign
DetailedDesign
Implementation
Operations &Maintenance
SystemAcceptance
SubsystemVerification
Integration &Test
Time
Relates to
Relates to
Relates to
Advanced Planning of Verification (Inspection,
Analysis, Demonstration, Test) Methods Introduced to work on the right
side of the “Vee”
2828© 2014 Noblis, Inc. Noblis proprietary and confidential.
Introduced standardized test procedures
Problem: Planning for verification was limited, resulting in inconsistent testing for conformance
Solution: Provides a requirements to test case/test procedure matrix Provides IEEE std 829-1998 guided test case/test procedures Standardized test procedures verify conformance Implementations use the test procedures; results are consistent
“… The testing team was able to quickly identify problems and assign responsibility. This process fostered an amicable environment between the agency and the vendor which produced very fast resolution to problems.”
Ashwin AmannaVirginia Tech Transportation Institute
Deployment and Testing of an Updated Dynamic Message Sign (DMS) Standard, FINAL REPORT, April 24, 2007
2929© 2014 Noblis, Inc. Noblis proprietary and confidential.
Requirements to Test Case/Procedure Martix
By planning the testing out, local agencies were assured that the system was verified.
Requirements to Test Case & Test Procedure Matrix
Requirement Test CaseID Title ID Title
3.5.3.1.1.3 Execute Climate-Control Equipment Testing C.3.5.3
C.3.5.4
Climate-Control Equipment Test - No ErrorsClimate-Control Equipment Test - Errors
3030© 2014 Noblis, Inc. Noblis proprietary and confidential.
Standardized test procedures
Test Case & Test Procedure Consistent Formats
Step Test Procedure ResultsAdditional References
1 CONFIGURE: Determine the event class to utilize for this test (e.g., per the test plan). RECORD this information as: »Class_Index
16 GET the following object(s): »eventConfigStatus.Event_Index
Pass / Fail(Section 3.4.2.2)
Section H.3.1.2Step d
17 VERIFY that the RESPONSE VALUE for eventConfigStatus.Event_Index is equal to ‘log’ (3). NOTE--Valid enumerated values for eventConfigMode are defined in NTCIP 1103, Section A.7.5.1.9 (Event Log Configuration Status Parameter).
Pass/Fail (Section 3.4.2.2)
Includes test descriptions, identification of variables, test case results, etc.
3131© 2014 Noblis, Inc. Noblis proprietary and confidential.
Summary
Systems engineering ■ Formalizes ITS Standard development in three stages (ConOps,
requirements, design)
■ Ensures quality in deployable standards that support interoperability and interchangeability
■ Provides for Verification & Validation of the standard at each stage
■ Combines systems engineering expertise with existing domain expertise, as best practices recommend
The new ITS Standards, developed using systems engineering concepts make it easier to procure and test
3333© 2014 Noblis, Inc. Noblis proprietary and confidential.
Additional References
Applying Systems Engineering Principles to the Development of Transportation Communication Standards - See more at: http://www.standards.its.dot.gov/LearnAboutStandards/SystemsEngineering#sthash.dgcraB3G.dpuf
The systems engineering process is used during the development of ITS standards - See more at: http://www.standards.its.dot.gov/LearnAboutStandards/SystemsEngineering#sthash.yj2xUrat.dpuf
The ITS Standards Program is managed and funded by the United States Department of Transportation (USDOT), Intelligent Transportation Systems Joint Program Office (ITS JPO), Steve Sill is the Program Manager and can be contacted at 202-366-1603 or Steve.Sill@dot.gov
USDOT ITS Standards Website http://www.standards.its.dot.gov/ For further information, contact:
• Blake Christie at blake.christie@noblis.org and 202-488-5711
• Ann Diephaus at ann.diephaus@noblis.org and 202-488-5718