Advancing Government through Collaboration, Education and Action
Cloud Community of Interest
FedRAMP Accelerated Feedback
Joint Authorization Board (JAB) Prioritization
Government – Cloud Service Provider (CSP) – Third Party
Assessor Organization (3PAO)
Release Date: October 18, 2016
Advancing Government through Collaboration, Education and Action 2
FedRAMP High-Level Prioritization Criteria
Advancing Government through Collaboration, Education and Action
FedRAMP High-Level Prioritization Criteria
• Purpose: Ensure that cloud service offerings selected for Joint
Authorization Board (JAB) Provisional Authority to Operate (P-ATO)
meet U.S. government’s (USG) cloud strategy and FedRAMP strategy.
• Outcome: JAB identifies the highest priority cloud service offerings
the JAB must authorize.
• Assumptions:– FedRAMP Director will grant a CSP the status of “FedRAMP Ready”
for the cloud service offering that demonstrates the ability to
successfully complete the JAB authorization process and have an
acceptable risk posture, as defined by the FedRAMP Readiness
Assessment.
– The number of FedRAMP cloud service offerings selected will be in
accordance with the capacity of the JAB.
3
Advancing Government through Collaboration, Education and Action
FedRAMP High-Level Prioritization Criteria
(continued)
• Current United States Government (USG) Market Use and
Demand
– Current usage of CSP offering:These CSP offerings in-use at multiple Agencies will measure the potential Return on
Investment (RO)I for a given JAB P-ATO. Identified by # of CSP offerings:
• In use at more than one Agency with an Agency ATO
• With a current Agency ATO interested in transitioning to JAB P-ATO
• Used across Agencies not reported and that FedRAMP has no ATO on file for the
CSP offering
– Demand for a CSP offering: number of Agencies that have expressed interest in a CSP
offering that have not yet received an ATO
• Impact to USG – Watermark and Innovation
– 70% of government cloud systems are at the “moderate watermark” per National
Institute of Standards & Technology (NIST) definition
– Highly innovative cloud products that may offer potential cost savings
4
Advancing Government through Collaboration, Education and Action
FedRAMP High-Level Prioritization Criteria
(continued)• Confidence and Commitment of CSP Excellence
– Organizational Maturity
• Understanding of FedRAMP and requirements
• Systems that belong to mature organizations with processes in place
– Capability Maturity Model Integration (CMMI) Level 3+
• Systems with very low risk associated with critical capabilities review
• Corporate culture with IT security in mind, previous audits available
– Project Management
• Timeliness of Deliverables, Deadlines (self-imposed/FedRAMP required) are
met
• Established communication channels with FedRAMP Program Management
Office (PMO)Resources allocated and committed to project
– CSP Dedication to Excellence
• Relationship building/established
• CSPs desire and commitment to JAB path
5
Advancing Government through Collaboration, Education and Action
FedRAMP Accelerated Feedback
JAB Prioritization – Government
Prema Nair, NIH
Ann Marie Keim, NASA, Sharon Ehrenberg, VA
Cloud Community of Interest
6
Advancing Government through Collaboration, Education and Action 7
• Federal Market
– Generate an increased interest for suppliers to obtain
FedRAMP authorization.
• Benefits
Streamline testing analytics
Opportunities for innovation
Contribute to RFIs
Accelerate award process
ROI
JAB Prioritization
Advancing Government through Collaboration, Education and Action 8
• Areas of consideration
– Category Deployment Models to increase similar product
interest and opportunity methods desired by agencies in:
Public Cloud
Community Cloud
Hybrid Cloud
Private Cloud
[But Private Cloud not a consideration for JAB path,
just agency authorization path?]
JAB Prioritization
Advancing Government through Collaboration, Education and Action
Opportunity Tools
• Federal Business Opportunities (FBO)
– Utilizing the search feature for Cloud allows for a
snapshot of current requirements
– https://www.fbo.gov/
• Federal Procurement Data System (FPDS-NG)
– Utilizing the reports feature for Cloud provides previous
award history for all federal agencies
– https://www.fpds.gov/fpdsng_cms/index.php/en/
9
Advancing Government through Collaboration, Education and Action
Training methods
• Current knowledge/training methods for Cloud
guidance and options
National Defense Industry Association (NDIA) – central
scheduling and marketing of events specific to Cloud
General Services Administration (GSA) - gsa.gov/cloud
Department of Defense (DoD)
Department of Homeland Security (DHS)
National Aeronautics & Space Administration (NASA) Solutions
for Enterprise Wide Procurement (SEWP)
FedRAMP.gov
Private Industry
10
Advancing Government through Collaboration, Education and Action 11
• Existing strong commercial capability offering
New to the market, or ground-breaking new capability to the
Federal market - may find this entry harder.
So many companies out there do not want to go through the
hoops –expense/time of FedRAMP.
How can we make it attractive for these companies to bring their
capabilities into the federal market?
This would help with the need to have more of a selection of
similar product.
Otherwise if restricted to FedRAMP authorized only, amounts to
sole source, so there needs to be more competition.
Market penetration in the Federal market
Advancing Government through Collaboration, Education and Action
Company with Previous Federal Capabilities
• Prioritization for an established company providing
assurance they will be around for more than a few
years.
• Historical validity and stability of a company, or
maturity model.
• However does this present a barrier to new
businesses and new capabilities or competition?
12
Advancing Government through Collaboration, Education and Action
Possible Exit Strategy for Non-Compliant CSPs
• JAB to de-prioritize CSPs that are not maintaining their
security posture
– Offload to agency if they have one as a customer for
Continuous Monitoring.
– Or if no agency, held to an Improvement Plan with a 3 to
6 month ‘graduation’ before revoking their authorization.
• More available time for JAB as a result.
• Possible Criteria:
– Patching, end of life support for hardware or software.
– Data breaches, lack in what they are doing to address.
them.
13
Advancing Government through Collaboration, Education and Action
JAB Prioritization Recommendations
• Have an Inter-Agency board Advisory Committee
– Consisting of small, medium, and large agencies with
varying cloud experience – expert to less experienced.
– 7 to 9 members with an Office of Management & Budget
(OMB) MAX page for collaboration.
– To give input into FedRAMP of what types of services
would be useful/desirable, possibly already are in
commercial but not yet in Federal for prioritization.
– Committee to also track other important inputs such as
Cloud Computing Summit, Cloud Security Alliance, and
others.
14
Advancing Government through Collaboration, Education and Action
JAB Prioritization Recommendations
• Procurement requirements should be considered.
• Prioritize on CSP that have contractual security
requirement language with acquisitions or third
parties / have been audited.
• Contract procurement language should reflect
FedRAMP requirements to be considered.
15
Advancing Government through Collaboration, Education and Action
FedRAMP Accelerated Feedback
JAB Prioritization – CSP
Bobbie Browning, Browning Partners
Nate Johnson, Microsoft
Cloud Community of Interest
16
Advancing Government through Collaboration, Education and Action 17
Prioritization Steps
1) Ability to achieve FedRAMP Ready status
2) Illustrate Cloud-service testing
3) Accommodate innovation and Small or Socio-
economically designated CSP solution
– Necessary services
– Competitive services
– Compelling technologies
– PMO “Shark Tank” Capability Interview to contribute to
final priority
Criteria for JAB Prioritization
Advancing Government through Collaboration, Education and Action 18
• Evaluated by the 3PAO as part of FedRAMP ready
process
• Must be completed and of adequate quality for
assessment completion
• Provides an additional gating function to ensure CSPs
are ready to move forward with assessment and
prioritization after FedRAMP Ready determination
Evidence of completed documentation
Advancing Government through Collaboration, Education and Action
Illustrate Cloud-service testing
• Commercial CSP proven in another vertical
• Document compliance in CSP Questionnaire
• Aligns with Federal priorities: OMB, Chief
Information Council (CIO) Council etc
– Financial - highly resilient and required
– Infrastructure and/or security > fundamental to federal
– Human Resources (HR) > employee engagement
– Retail > citizen and beneficiary services
19
Advancing Government through Collaboration, Education and Action
Accommodate Innovation and Small
or Socio-economically designated CSP
• Adopt a Portfolio Management strategy
– Necessary services
– Competitive services
– Compelling technologies
• Establish PMO “Shark-Tank” Capability Interview to
contribute to final priority
• Incorporate consideration for small businesses in
the scoring of these Capability Interview CSP
presentations
20
Advancing Government through Collaboration, Education and Action
Criteria to participate in Accelerated JAB
Provisional Authorization to Operate (P-ATO)
1) Create qualification scheme template
2) Establish a weighted-score method to rank CSPs
3) Share expectations to participate in JAB-P-ATO
4) Direct CSP to complete Questionnaire
5) Apply weighted-score to completed Questionnaire
6) Rank CSPs to determine readiness for FedRAMP Ready
Audit
7) Schedule 3PAO to perform FedRAMP Ready Audit
8) Identify gaps, determine & communicate next step
9) Obtain CSP commitment to satisfy gaps
21
Advancing Government through Collaboration, Education and Action
Criteria to participate in Accelerated JAB P-ATO
(continued)
• FedRAMP Ready – CSP get a 3PAO & validate
– 3PAO assessment of rated maturity – (Rank 1-5)
• Evidence of quality and cloud testing
– Documentation package readiness and quality (Rank 1-
5)
• PMO/JAB initial review of quality
– Shark-Tank Capability Interview - CSP defend (Rank 1-5)
• Capability needed – Based on demand research: RFI’s, Sources
Sought, and any other services responsive to agency demand
• Ground-breaking capability matching Govt Directives
• Introduces competition into the market of existing capabilities
being used
• Commercial CSP proven in another vertical with capabilities that
the govt doesn’t yet know they need 22
Advancing Government through Collaboration, Education and Action
Prioritization Score Example
23
43
5
FedRAMP Ready Maturity(3PAO assessed)
Documentation Quality Capability Interview
Total CSP Score = 12 out of possible 15
Advancing Government through Collaboration, Education and Action
Potential CSP Questionnaire
• How is your organization structured to pursue a JAB P-ATO
and maintain it operationally, once achieved?– Dedicated security officer?
– HR security training for all new employees and process to recertify annually?
– Sales trained, and with what frequency, to field questions from customers
– Additional comments?
• What P&P demonstrate your organizational maturity?– Do you have a ticket system to document every system change?
– Describe how, and with what frequency, you establish and communicate
priorities across the organization.
– How do you validate all stakeholders are aligned with those priorities?
• How would you describe the resiliency of your infrastructure?
– Disaster Recovery
– Redundancy
– Monitoring Systems
– System Scans
24
Advancing Government through Collaboration, Education and Action 25
• Adopt a Portfolio Management strategy
• Allocate rotating PMO resources for quarterly, day-long
prioritization sessions to establish the CSP queue
• Include Agency reviewers in CSP presentations
• Conduct 15 – 30 minute CSP presentations similar to VC:
– Solution capabilities
– Market diversification
– Government market penetration – not a requirement
– Security compliance experience
• Caucus reviewers immediately following presentation for an
up/down decision to proceed to next step
• Notify CSP of disposition within 2 weeks of presentation
• Leverage “FedRAMP High-Level Prioritization Criteria”
PMO “Shark-Tank” Capability Interview
Advancing Government through Collaboration, Education and Action
Considerations to Increase Success of CSPs
• Communicate expectations to CSP
– Rigor & benefits of a JAB Technical Review
• Access to compliance experts
• Positive operational impact: “smart stuff” in detailed requirements
– Initial investment in infrastructure and resources -
dependent upon:
• Existing resilience at beginning of the process
• Approach to satisfying requirements e.g. redundancy
– Recurring investments – one CSP example:
• Operations – 2 Full Time Employees (FTEs)
• Security – 1 FTE
26
Advancing Government through Collaboration, Education and Action
Considerations to Increase Success of CSPs
(continued)• Supplement PMO resources
– Recommend CSP hire 3PAO Consultants > ROI
– Establish a CSP Mentor Program – “If only we had known. . .or had an
experienced CSP to ask. . .”
• Absence of clarity
• Subject to many judgment calls
• Lessons learned from a (Software as a Service) SaaS CSP
about the JAB process
1) Get the whole village engaged
2) Get organized, very, very organized
3) Embrace the change
4) Get help
5) Educate your Federal customers
6) The FedRAMP JAB P-ATO is the beginning of something, not the end
27
Advancing Government through Collaboration, Education and Action
Recommendations for PMO Resources
• Allocate PMO resources to accommodate
Continuous Monitoring and new certifications
– Maintain existing CSPs – ConMon: 50%
– Support 8 to 12 new certifications annually: 30%
– Rotate PMO staff through Shark-Tank reviews 20%
• Identify criteria for participation in Shark-Tank
– Create a qualification scheme
– CSP complete questionnaire that is quantifiable
– Establish a score that illustrates ranking
• Assess ability to impact the competitive landscape
• Consider dynamic allocation of PMO resources to
respond to external pressures that change priorities 28
Advancing Government through Collaboration, Education and Action
FedRAMP Accelerated Feedback
JAB Prioritization – 3PAO
Maria Horton, Emesec
Abel Sussman, Coalfire
Cloud Community of Interest
29
Advancing Government through Collaboration, Education and Action
JAB Prioritization 3PAO Recommendation(s)
• Objective:
– Identify 3PAO community Recommendations on
concrete criteria for CSP designation as a JAB
Prioritized Solution
• Detail the methodology for designation
• Methodology of Working Group
– Focus group discussions from a variety of 3PAOs
• Self selected group –
• Must be members of ACT-IAC
– Outside vetting may enhance receptivity by cloud
community
30
Advancing Government through Collaboration, Education and Action
Assumptions
• JAB Prioritization 3PAO Working Group
recommendations to be shared and vetted for input
or feedback with:
– Government Working Group
– CSP Working Group
• Presentation to GSA FedRAMP PMO
– Potential input from FedRAMP PMO may alter
recommendations:
• JAB timeline of “prioritized evaluation” (is this hard or soft?)
– First in- First out to evaluation? Or other commitment schedule
• Non-punitive follow up to “prioritized” designees for JAB Board
31
Advancing Government through Collaboration, Education and Action
Prioritizing Innovative & Necessary Services
• Different ways to define innovative
– May reflect risks, new goals, etc. More likely to be one-of needs by
CIOs
– Question: How do we acknowledge innovative services that
government does not know it needs (yet) and prioritize without
politicizing?
• Example may be a Cyber Security Prevention Tool for Terrorist events
• Necessary services interpreted differently by Agency
– Quantity needs may be seen as necessary such as sought-after tools
echoed by many agencies
• Recommend: The number of needed Agencies that reflect broad use be
defined and publicized
– Question: How many agencies are required to equate broad
Government use?
32
Advancing Government through Collaboration, Education and Action
Factor 1: Timing for CSP Participation
• Recommend development of “Cohorts” - a wave or phased
release of those submitted into the accelerated program
– Timing of the cohorts are not defined but 2-4 releases per year
seems to be reasonable
– Few cohorts in the beginning to pilot program and gain feedback
– CSPs should know the cohort timings and criteria for submission and
for review.
• Understand no reprisals for not being selected or later moving out of the program
• Open Questions:
– How do we determine the number of CSPs selected for the Cohort?
– Do all Cohort slots need to be filled?
• Retain openings if CSPs do not meet certain threshold
• Reserved openings for small business, congressional input, etc
33
Advancing Government through Collaboration, Education and Action
Factor 1: CSP Selection Criteria
• Completed FedRAMP Ready
– No measurable criteria other than FedRAMP Ready testing is
currently done
– Maturation of Security Program (May submit a mix) as determined by
FedRAMP Ready designation of Level I-V
• Category of services
– Determined by Government strategic needs/CIO Council
– Examples
• Infrastructure
• Admin / Back Office
• Apps for Gov’t Employees
• Apps for Citizens/Beneficiaries
• Security?
34
Advancing Government through Collaboration, Education and Action
Factor 2: Selection Variables
• “Clean” or Complete documentation
– If quality of paperwork is an indicator, this should become
a FedRAMP Ready element
• Not currently part of FedRAMP Ready - just the technical
evaluation
• Contractor size or socio-economic designations
• Unique Congressional interest
• Other variables?
35
Advancing Government through Collaboration, Education and Action
Sample
• Cloud Solution
– Identified in one or more Agency Strategic Plan
– Has one or more Capital Expenditures (CPIC or OMB
Exhibit 300’s) dated within the last 2 years
• Makes it current and not a long standing issue
– The CSP has been in business at least 24 months
• Goes to stability
• Is in commercial use -
– The FedRAMP Ready evaluation has been completed
within the last 6 months
36
Advancing Government through Collaboration, Education and Action
Evaluating the Selection Criteria
• Selection criteria for Prioritized CSPs need::
– Need to be quantitative in nature
– Clearly defined criteria to prevent bias perception
• Selection criteria and weights assigned can change (per fiscal year or
cohort) to reflect Government needs
– An example of the variable / weighting system follows:
37
CSP Solution 1
Score Weighting Total
(1-5)
Criteria 1 1 20% 0.2
Criteria 2 3 30% 0.9
Criteria 3 4 10% 0.4
Criteria 4 5 15% 0.75
Criteria 5 2 25% 0.5
2.75Total
CSP Solution 2
Score Weighting Total
(1-5)
Criteria 1 3 20% 0.6
Criteria 2 2 30% 0.6
Criteria 3 5 10% 0.5
Criteria 4 1 15% 0.15
Criteria 5 4 25% 1
2.85Total
Advancing Government through Collaboration, Education and Action
Marketing Suggestions for JAB Prioritization
• Specific criteria for prioritization must be publicized and achievable by all
businesses to demonstrate a balanced playing field
– Consider piloting the initial criteria variables and ask for industry feedback
• Defined Prioritization elements need to be measurable and possibly
identified in the FedRAMP Ready process.
– Would this allow just large companies to jump ahead since they may have
more funding and resources available?
– The more elements added to FedRAMP Ready, results in a bigger hurdle for
small businesses and new Federal businesses
– Cause and Effect: Likely to result in more CSPs needing to complete all
documents prior to FedRAMP Ready
38
Advancing Government through Collaboration, Education and Action
Open Questions
• Customers talk about the cost of compliance, and how do we equate this to ROI? How can we articulate
this?
• Can CSPs leverage 18F and/or Compliance Masonry to streamline documentation maintenance
(ultimately allowing them to reallocate resources to technical maintenance for Continuous Monitoring)?
• If a CSP is selected for JAB prioritization because they have demonstrated maturity in through
FedRAMP Ready, and the CSP isn’t ready to present to the JAB in a year, what happens?
– Is there or will there be an extension or waiver process? Are they “pushed” back to regular process?
• Should we limit categories or types of solutions will be included within the priorities?
– Recommend limiting IaaS to 1 per year -- several already exist
– Recommend how to identify the unique needs that are a result of current timing
– Determine if at least 2 competitors should be JAB ready for those unique innovations
• How do we address the complaints, accusations or inherent barriers to small business and potential
innovators?
– Each step adds overhead costs to entering the Federal Market
– Plan in advance how to address Congressional Inquiries
– Determine if 1-2 slots are left for those businesses using Congressmen to push from their local areas
– How to eliminate politics if this isn't what the JAB should handle
39
Advancing Government through Collaboration, Education and Action
End of Presentation – Open Discussion
40
Advancing Government through Collaboration, Education and Action
Thank You to Our Contributors
Doug Noakes, Booz Allen Hamilton; Maria Horton,EmeSec; Abel Sussman,
Coalfire; Bruce Hamilton, EY; Daniel Lee, Censeo Consulting; Kyle
Hendrickson,BRMi; Richard Beutel,Cyrrus Analytics; Saif Rahman, Quzara;
Bobbie Browning, Browning Partners; Ken E. Stavinoha, Ph.D, CISCO
Systems; Marilyn Hays, HPE; Brian Cram,IBM; Erica Poskaitis, Oracle;
Stacy Cleveland, HPE; Nate Johnson, Microsoft; Eric Adams, IBM; Prema
G. Nair, NIH; Phillip D. Dixon, Dept of Labor; Michael Christopherson, GSA;
John Frary, CSRA; Michael Cassidy, DOJ; Sharon Ehrenberg, VA; Ann
Marie Keim, NASA; Roopangi Kadakia,VA; Monette Respress,Noblis
41