Think Differently: Cybersecurity and the
Adaptive Acquisition Framework
Securing the DoD Supply ChainCybersecurity Maturity Model Certification
Ms. Katie ArringtonChief Information Security Officer for Acquisition
2
UNCLASSIFIED
UNCLASSIFIED
CMMC Model Structure
3
17 Capability Domains (v1.0) Capabilities are assessed for Practice and Process Maturity
DISTRIBUTION A. Approved for public release
UNCLASSIFIED
4
LEVEL 1BASIC CYBER HYGIENE
LEVEL 2INTERMEDIATE CYBER HYGIENE
LEVEL 3GOOD CYBER HYGIENE
LEVEL 4PROACTIVE
LEVEL 5ADVANCED / PROGRESSIVE
17 PRACTICES
Demonstrate compliance with Federal Acquisition Regulation (FAR) 48 CFR 52.204-21
72 PRACTICES
Comply with the FAR
Perform a select subset of 48 practices from the NIST SP 800-171 r1
Perform an additional 7 practices to support intermediate cyber hygiene
130 PRACTICES
Comply with the FAR
Perform all 110 practices from the NIST SP 800-171 r1
Perform an additional 20 practices to support good cyber hygiene
152 PRACTICES
Comply with the FAR
Perform all 110 practices from the NIST SP 800-171 r1
Perform a select subset of 13 practices from Draft NIST SP 800-171B
Perform an additional 29 practices to demonstrate a proactive cybersecurity program
171 PRACTICES
Comply with the FAR
Perform all 110 practices from the NIST SP 800-171 r1
Perform a select subset of 17 practices from Draft NIST SP 800-171B
Perform an additional 40 practices to demonstrate an advanced cybersecurity program
CMMC Practice Progression
DISTRIBUTION A. Approved for public release
Reduces risk of Advanced Persistent Threats (APTs)
UNCLASSIFIED
5
LEVEL 1PERFORMED
LEVEL 2DOCUMENTED
LEVEL 3MANAGED
LEVEL 4REVIEWED
LEVEL 5OPTIMIZING
0 PROCESSES
Select practices are documented where required
2 PROCESSES
Each practice is documented, including Level 1 practices
A policy exists that includes all activities
3 PROCESSES
Each practice is documented
A policy exists that includes all activities
Adherence is verified through Examine or Test
A plan exists, is maintained, and resourced that includes all activities (includes mission, goals, project plan, resourcing, training needed, and involvement of relevant stakeholders)
4 PROCESSES
Each practice is documented
A policy exists that includes all activities
Adherence is verified through Examine or Test
A plan exists that includes all activities
Activities are reviewed and measured for effectiveness (results of the review is shared with higher level management and for issue resolution)
5 PROCESSES
Each practice is documented
A policy exists that includes all activities
Adherence is verified through Examine or Test
A plan exists that includes all activities
Activities are reviewed and measured for effectiveness
There is a standardized, documented approach across all applicable organizational units
CMMC Maturity Process Progression
DISTRIBUTION A. Approved for public release
UNCLASSIFIED
CMMC Model Evolution v0.4 to v0.5 to v0.6 to v0.7 to v1.0
6
380
859
316
59 9
219
44 9
17343 9
17143 5 35
11592 96
4233
78 83 85
3717
58 56 62
2617
55 59
26 1617
55 58
26 15
Level 1 Level 2 Level 3 Level 4 Level 5
Practices by Level
40
19
26
1621 21
17
41
913
5
17
8
36
16 17
45
13
39
1720
16 18
7
16
30
9 10
3
128
27
16 16
40
12
34
5
15
5
15
0
16 18
7 94 6 4
1511
5
35
15
26
3
14
511
0
1114
6 82
6 4
128
3
27
13
26
2
14
511
0
11 13
6 82
6 4
128
3
27
13
AC AM AA AT CM CG IDA IR MA MP PS PP RE RM SAS SA SCP SII
Practices by Domain
V0.4 V0.5 V0.6 V0.7 V1.0
DISTRIBUTION A. Approved for public release
UNCLASSIFIED
• CMMC Model leverages multiple sources and references– CMMC Level 1 only includes practices from FAR Clause 52.204-21
– CMMC Levels 4 and 5 do not include QTY 16 practices from Draft NIST SP 800-171B because of implementation complexity/constraints and/or cost
CMMC Model v1.0: Source Counts
7
CMMC Model v1.0: Number of Practices per Source
* Note: QTY 15 safeguarding requirements from FAR clause 52.204-21 correspond to QTY 17 security requirements from NIST SP 800-171r1, and in turn, QTY 17 practices in CMMC
CMMC Level
Total Number Practices per CMMC Level
Source
48 CFR 52.204-21
NIST SP 800-171r1
Draft NISTSP 800-171B Other
Level 1 17 17 * 17 - -
Level 2 55 - 48 - 7
Level 3 58 - 45 - 13
Level 4 26 - - 13 13
Level 5 15 - - 4 11
Total 171 17* 110 17 44
Excluded - - - 16 -
DISTRIBUTION A. Approved for public release
UNCLASSIFIED
Grant Certification
Conduct Certification
Certificate
Update
Internet Accessible Lookup
Advance to Level
Options:1. Internal2. SVC Provider3. Partner
Source Selection
(Go/No-Go)
RFP Award
Self-Evaluate
Companies Create
DatabaseEst. PMO
Office
ACQ Review
RFI “Level x”& Date
Develop Model
CMMC Concept
CMMC REQT
PMRequiring Activity
Select Certifier
CertifierDevelop
Accreditation Body REQT.
Est. MOU Accrd. Body
BID
Verify CMMC Level
FindCertifier
Document Cert
Accreditation BodyCMMC Gov’tGov’t PMCertifierCompany
SRM Database
Sr. Advisory Council
Beginwork
Accrd. BodyIOC
CMMC Implementation Flow
BeginWork
Accrd. BodyIOC
Market Place
CMMCCertificateDatabase
Create Databas
e
8DISTRIBUTION A. Approved for public release
UNCLASSIFIED
CMMC Accreditation Body Activities
9
Accreditation Body (AB) Manager
Training Accreditation
Credentialing
Infrastructure
(Support Systems)
• Train Individuals
• Train Organizations
• Train Instructors
• Knowledge Store
• Market Place • Artifact Store• Records
Mgmt.
• Grant C3PAO accreditations
• Audit C3PAO• Process Complaints
• Grant Individual credentials
• Certifiers• Accredited
Certifiers
• Coordinate w/ CMMC PMO and CMMC Advisory Council
• Dispute resolution• Capture metrics• Integrate and coordinate
functional areas
Assessment Operations
• Technical Appeals• Quality Control• Manage
Assessment Tool• Publish CMMC
Certificates
Populated and accessible by DoD
systems
CMMCDatabase
DISTRIBUTION A. Approved for public release
UNCLASSIFIED
CMMC Draft Schedule: CY20
10
Q2FY20 Q3FY20 Q4FY20 Q1FY21
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Rulemaking
CMMC Roll-Out Plan
DoDI 5000.02 Cybersecurity Enclosure
CMMC Implementation Pathfinder(s) with Subset of DIB Sector
CMMC Accreditation Body (AB)
CMMC Databases & Infrastructure
CMMC AB Training• Train the trainers • CMMC classes for assessors
Draft CMMC Training Material to CMMC Accreditation Body (AB)
Defense Acquisition University (DAU) CMMC Training
CMMC Assessment Guides
CMMC Model
DAR Council Meeting Goal: Complete Rulemaking Process
Initial RFIs with CMMC Requirement
Initial RFPs with CMMC Requirement
* Depends upon Rulemaking
Complete reviews and approval
Complete & Release v1.0
Update & Refine CMMC Assessment Guides
Deliver Levels 4-5 to CMMC ABDeliver Levels 1-3 to CMMC AB
Deliver Draft TrainingCMMC 101 Coordinate and Conduct Training Pathfinder with CMMC AB
Deliver Draft TrainingCMMC Levels 1-3
Deliver Draft TrainingCMMC Levels 4-5
Establish CMMC AB Board
Sign MOU (TBD) Marketplace (TBD)
Certification process for candidate CMMC Third Party Assessment Organizations (C3PAOs)
Initial Planning Database/Infrastructure Pathfinder
Initial Beta Testing
Initiate Training for CMMC 101, Levels 1-3
Initiate Training for CMMC Levels 4-5
Refine Draft Training & Conduct Training Pathfinder with DoD
Initiate Training for CMMC 101, Levels 1-3Initiate Training for CMMC Levels 4-
5
* Depends upon Rulemaking
Potential update based on rulemaking process (TBD)
Pathfinder(s)Initial Planning
UNCLASSIFIED
Projected CMMC Roll-Out
11
Total Number of Prime Contractors and Sub-Contractors with CMMC RequirementFY21 FY22 FY23 FY24 FY25
Level 1 895 4,490 14,981 28,714 28,709
Level 2 149 748 2,497 4,786 4,785
Level 3 448 2,245 7,490 14,357 14,355
Level 4 4 8 16 24 28
Level 5 4 8 16 24 28
Total 1,500 7,500 25,000 47,905 47,905
Total Number of Contracts with CMMC RequirementFY21 FY22 FY23 FY24 FY25
15 75 250 479 479
• OUSD(A&S) will work with Services and Agencies to identify candidate programs that will have the CMMC requirement during FY21-FY25 phased roll-out
• All new DoD contracts will contain the CMMC requirement starting in FY26
UNCLASSIFIED
DISTRIBUTION A. Approved for public release
12
https://www.acq.osd.mil/cmmc/index.html
DISTRIBUTION A. Approved for public release
UNCLASSIFIED
Douglas Hubbard Author and founder of Hubba rd Decis ion Resea rch (HDR)
Hubbard Decision Research2 South 410 Canterbury CtGlen Ellyn, Illinois 60137
www.hubbardresearch.com
© Hubbard Decision Research, 2020
How to Measure Anything in
Cybersecurity Risk
What is your single biggest risk in cybersecurity?
How you assess cybersecurity risk.
15
Summarizing Research on Risk Matrices
◇ “Risk Matrices should not be used for decisions of any consequence.”■ Bickel et al. “The Risk of Using Risk
Matrices”, Society of Petroleum Engineers, 2014
◇ “…they ca n be ‘w orse tha n useless’”■ Tony Cox “W ha t’s w rong w ith Risk
Ma trices” inves tiga tes va rious ma thema tica l consequences of ordina l sca les on a ma trix.
16
Like
lihoo
d
Impact
“The first principle is that you must not fool yourself, and you are the easiest person to fool.”
—Richard P. Feynman
17
Analysis Placebo An apparently “structured” method will increase
confidence in estimates and decisions even when measured performance is the same or worse.
What Works?
Research Results:
Simple statistical models beat human experts in a wide variety of estimates
and forecasts.
18
Paul Meehl assessed 150 studies comparing experts to statistical models in many fields (sports,
prognosis of liver disease, etc.).
Philip Tetlock tracked a total of over 82,000 forecasts from 284
political experts in a 20 year study covering elections, policy effects, wars, the economy and
more.
What Quantitative Risk Can Look Like
19
Inherent Risk
Risk Appetite
Residual Risk
Loss Exceedance Curve
Probability-Weighted Average
Return on ControlControl 1 846%Control 2 131%Control 3 15%Control 4 -45%
How much risk do we have? Is the risk acceptable? How should I reduce risk?
A Simple “One-For-One Substitution”
Each “Dot” on a risk matrix can be better represented as a row on a table like this.
The inputs are used in a Monte Carlo simulation in Excel.
The output can then be represented as a Loss Exceedance Curve.
20
Examples can be found at www.howtomeasureanything.com
A Path to Improvement
21
Even without new data, some subjective estimation methods are better than others.
To improve estimates, you have more data than you think
You need less data than you think
A very mathematical corollary from super advanced statistics:
If you know almost nothing, almost anything will tell you something
LCDR Ryan HilgerUSN
Big IdeaIn an increasingly complex threat environment,
a cquis ition profess iona ls need: ◇ a better unders ta nding of their progra m’s
a tta ck surfa ce a nd ◇ a different w a y to conceptua lize risk a nd
mea sures to mitiga te it
All statements are made in a personal capacity and do not represent the views of Navy Strategic Systems Programs, the Department of the Navy, or the Department of Defense
Expanding on the Big Idea ◇ A complete decomposition of the problem statement unlocks new
options■ More alternatives than technological solutions!■ Understand what information would cause you to change the
program cost/schedule/performance baseline◇ A well-grounded understanding of the theory behind the system is
crucial to effective operations■ Overall, most seem to lack sufficient understanding of probability,
statistics, and data curation to effectively implement this paradigm◇ Healthy skepticism for even long-standing tools and processes improves
outcomes
All statements are made in a personal capacity and do not represent the views of Navy Strategic Systems Programs, the Department of the Navy, or the Department of Defense
Takeaways◇ Develop. Your. People. They are our most important asset. Treat them as
such.◇ Multiple options exist to get help:
■ GSA Schedule contracts■ Small Business Innovation Research contracts■ Simplified Acquisition contracts
◇ It is worth a bit of program budget to learn!
All statements are made in a personal capacity and do not represent the views of Navy Strategic Systems Programs, the Department of the Navy, or the Department of Defense
Timothy DenmanCybersecurity Learning Director Defense Acquis ition Univers ity
27
“Adaptability” - the ability to effectively react to circumstances to reduce risk to cost, performance, and schedule
“Adaptive acquisition” • includes exposing outcome-based requirements to as broad a
marketplace of solution providers as possible; • benchmarking of best existing capability; • reactively adapting system design to take advantage of existing mature
technology; • and streamlining engineering, programmatic, and procurement
bureaucracy
• This begins with understanding risks and producing measurable and testable objectives.
Cybersecurity should be at the center of adaptive acquisition – it cannot be an afterthought
Adaptive Acquisition Framework DoDD 5000.01: The Defense Acquisition SystemDoDI 5000.02: Operation of the Adaptive Acquisition Framework
Tenets of the Defense Acquisition System1. Simplify Acquisition Policy2. Tailor Acquisition Approaches3. Empower Program Managers
4. Data Driven Analysis5. Active Risk Management6. Emphasize Sustainment
< 5 years
< 1 yearCybe
rsec
urity
PathSelection
Defense Business Systems
Middle Tierof
Acquisition
Acquisition of Services
Major Capability
Acquisition
OPE
RATI
ON
S AN
D SU
STAI
NM
ENT
Business Capability Acquisition Cycle
SoftwareAcquisition
RapidPrototyping
CapabilityNeed
IdentificationSolutio
nAnalysi
s
FunctionalRequirements and
Acquisition Planning
AcquisitionTesting and Deployment
CapabilitySupport
Plan
ning
Phas
e I1 I2…
MVP MVCR Rn
OD
Rapid Fielding
10
MaterialSolutionAnalysis
TechnologyMaturation
and Risk Reduction
Engineering and ManufacturingDevelopment
Production andDeployment
MDD MS A MS B MS C IOC FOC
ATP ATP ATP ATP
In In InExecution Phase
OD
< 5 years
UrgentCapability
Acquisition < 2 years
DD
1Formthe
Team
2ReviewCurrentStrategy
3PerformMarket
Research
4Define
Require-ments
5Develop
AcquisitionStrategy
6ExecuteStrategy
7Manage
Performance
PLAN DEVELOP EXECUTE
Legend:ATP: Authority to ProceedDD: Disposition DecisionFOC: Full Operational CapabilityI: IterationIOC: Initial Operational CapabilityMDD: Material Development DecisionMS: MilestoneMVP: Minimum Viable ProductMVCR: Minimum Viable Capability ReleaseOD: Outcome DeterminationR: Release
Version 3.2
Cybersecurity and DOT&E (FY18 Annual Report)DOT&E identified five improvement a rea s to ena ble cyber defenders to do their jobs w ell:
◇ Scope the ta sk by defining the key cyber terra in, opera tiona l miss ions , ta sks , a nd expecta tions .
◇ Foster unity of effort a mongst pa rticipa nts tha t ha ve different roles (offens ive, defens ive) a nd responsibilities (interna l a nd externa l to a ss igned key cyber terra in).
◇ Know the key cyber terra in, opera tiona l concepts , a nd a va ila ble tools .◇ Ma tch tools a nd skills to the opera tiona l ta sks , miss ions , a nd key
cyber terra in.◇ Pra ctice a nd tra in in opera tiona lly representa tive conditions a ga ins t
rea lis tic cyber-a tta cks .DOD missions and systems remain at risk from adversarial cyber operations. Significant improvements are being made, … BUT they are NOT outpacing the growing capabilities of potential adversaries.29
DAU learning offerings on Cybersecurity ◇ Expertise
■ 9 full-time cybersecurity professors■ 6 intermittent cybersecurity professors■ 6 locations spanning 4 time zones
◇ Customized Training Program DFARS CDI and Cybersecurity Maturity Model Certification (CMMC) v 1.0
◇ 24/7 Learning: www.dau.edu■ Online Courses ■ Cybersecurity Community of Practice■ On-the-Job Tools: Cybersecurity & Acq. Lifecycle Integration Tool (CALIT)
◇ Courses■ Cyber Training Range■ Capture the Flag and outreach events
◇ Consulting initiatives with all major Services and Cyber Table Top facilitation
◇ Credentialing: cybersecurity credential courses beginning in Spring 2020
◇ Townhalls, Workshops, Rapid Deployment Training
Delivering world-class cybersecurity training and consultingcontributing to a decisive edge for our warfighters30
31
32
Q & A
33
Upcoming Events
• Feb. 26: @4:00-5:30 pm ET @ The Garden: Actionable Inclusion Panel • Mar. 11: @4:00-5:00 pm ET @ The Garden: Suicide Prevention in the
DON
Follow the NavalX Eventbrite page for future events
Stay connected with NavalX
34
Upcoming Events
Feb. 12: DAU Webcast: Design Thinking on the Job Feb. 20: 2020 DAU Acquisition Update Feb. 26: DAU Webcast: Best Practices for Contracting for DMSMS
Management
DAU Events Calendar: https://www.dau.edu/EventsDAU webcasts: https://www.dau.edu/p/dau-webcasts
Stay connected with DAU