Date post: | 18-Jan-2017 |
Category: |
Technology |
Upload: | amazon-web-services |
View: | 737 times |
Download: | 0 times |
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Mark Johnston, Director of Global Business Development,
Healthcare and Life Sciences
May 3rd, 2016
AWS Healthcare DaysNashville, TN
Payers PatientsProviders
Health Information
Exchanges
Healthcare data
security Precision
medicine
Healthcare
ERPEHR
Revenue Cycle
Management
Connected Health
Ecosystem of established healthcare partners and new
entrants…..
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scott Whyte
SVP, Growth & Innovation - ClearDATA
May 3, 2016
Healthcare Cloud: Opening Remarks
AWS Healthcare Days | Nashville
Bad Day at the Datacenter
What I Hear…Often
“I think in 5 years, all providers will want to get out of the data center
business” - National Provider CIO
“I want my team to focus on innovation, not plumbing” – SaaS CTO
“We need competitive advantage - really fast” – Payer CTO
“We want to help providers take on risk – they need HIE and
analytics.” – Chief Analytics Officer, Payer
Agility
After moving to the cloud, Forbes found 60 per cent of
business leaders say they have reduced their IT
maintenance requirements, allowing them to focus more
on strategy and innovation, with 59 per cent seeing
increased business agility.
Community
Physicians
Participating
Practices and
Physicians
Quality Measures
Population management
Increased care coordination
Business model becomes more
focused on wellness
Financial Alignment
Shared risk/shared rewards
Cost reduction incentives
Shift from encounter-focus to
patient-focus
Clinical Integration Shared Services
Data
AcquisitionClinical Data
Repository
Extract clinical data
Extract claims data
Data
IntegrationPatient EMPI
Provider EMPI
Data Standardization
Quality Metrics
Analytics and
Reports
Health Team
CommunicationsPhysician
communication
Provider-patient
Provider-provider
Technology aspects are critical underpinnings to success
Clinician Knowledge
Find actionable activities (gaps)
Decision support
Enhance communications with
patients and other providers
Clinical
Integration
Solutions
Overview
Hospitals
Inpatient clinical
quality metrics
Payer
s
Physician-Led
Entities
Governing body (Participating
Practices and
Physicians)
Payer
negotiations
Distribute
shared savings
Clinical quality
Reports
Participating
Community
Physician clinical
data
Coordinated
Care
Collaboration
Innovation
Thank You.
Scott Whyte
SVP, Growth & Innovation - ClearDATA
Embracing DevSecOps while improving your
compliance and security agility and postureChris McCurdy
Healthcare and Life Sciences Specialist AWS
Agenda
• DevOps to DevSecOps Primer
• Observed industry cloud techniques with AWS• Tools, processes and frameworks to assist
• Example Compliance Workflows
Big Company, Big Challenges
Thousands of Systems
Complex IT Ops
Limited Financial Impact
Cloud Patterns and Acceleration
Automated IT Cost Transparency
Current State of Enterprise IT Cloud Strategy Offers Agility
DevOps Level Set
Development
Quality Assurance
Operations
DevOps
DevOps Toolchain
Plan
Configure
Verify
Preprod
Monitor
Create
Release
Define and plan; business value, application requirements and metrics
Building, coding and configuration
Ensuring quality; acceptance, regression testing
Infrastructure and application
Approval/certification, triggered releases, release staging and holding
Process, application and infrastructure
Release coordination, promotion, scheduling, rollback and recovery
DevOps Principles
• Collaborate with all stakeholders
• Codify everything
• Test everything
• Automate everything
• Measure and monitor everything
• Deliver business value with continual feedback
Manual Hacking
Drivers for DevSecOps
Embedding Security into DevOps was not successful because…
• Compliance checklists didn’t take us far before we stopped scaling…
• We couldn’t keep up with deployments without automation…
• Standard Security Operations did not work…
• And we needed far more data than we expected to help the business make decisions…
DevSecOps: Security as Code
Establishing these principles…
• Customer focused mindset
• Scale, scale, scale
• Objective criteria
• Proactive hunting
• Continuous detection and response
DevOps Toolchain
Plan
Configure
Verify
Preprod
Monitor
Create
Release
Define and plan; business value, application requirements, security, compliance
and metrics
Build, code and configuration
Ensuring quality; acceptance, regression, security and compliance testing
Infrastructure and application
Approval/certification, triggered releases, release staging and holding
Process, application, infrastructure, security and compliance
Release coordination, promotion, scheduling, rollback and recovery
Observed industry cloud techniques with AWS
Consult internally before implementing
The following slides are practices we
have seen used in industry. As security
and industry compliance is determined
by the customer before implementing
please:
• Consult with your internal best
practices
• Consult with with your Cloud Center of
Excellence
• Consult with your Information Security
group
• Consult with your Compliance
organization
• Do your due diligence
AWS Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure Regions
Availability Zones
Edge Locations
Cu
sto
mer
sPlatform, Applications, Identity & Access Management
Operating System, Network & Firewall
Customer content
Client-side encryption implementation, Server-side encryption, Network Traffic Protection
A Word on Security
Security
in the
cloud
Security
of the
cloud
Example: Simplified Claims Workflow
Validation
/ Edit
System
(EC2)
Insight
System
(EMR)
Inbound
Claim
Archive
(Glacier)
Inbound
Claim Store
(S3)
Claim History
(Redshift)
1
Claims
Adjudication
System
(EC2)
Data Lake
(S3)6
Insights
2 3 4
55
5
7
HIPAA Eligible
Architecture
Consult with compliance and security organizations before implementing
AWS Service
Amazon
EC2
Amazon
EMR
Amazon
GlacierAmazon
S3
Amazon
DynamoDB
Amazon
RDS (MySQL
and Oracle)
Amazon
Redshift
Amazon
EBS
Elastic Load
Balancing
Amazon ECS AWS Elastic
Beanstalk
AWS
CodeCommit
AWS
CodeDeploy
AWS
CodePipeline
SQSSNS
AWS Config
AWS
Device Farm
AWS HIPAA Eligible Services (as of 4/21)
AWS Non-HIPAA Eligible Services
Consult with compliance and security organizations before implementing
General Strategies
AWS
CodeCommit
AWS
CodeDeploy
AWS
CodePipeline
Consult with compliance and security organizations before implementing
• Decouple PHI data from the processing
or orchestration
• Do not check PHI data into your source
or artifact repositories
• Use indirection when orchestrating PHI
flow
• Separate PHI and non-PHI containing
logical boundries
• Monitor the flow of PHI
Separate Virtual Private Cloud (VPC) Strategy
Amazon
EC2Amazon
EMRAmazon
S3
PHI Eligible VPC
Amazon
EC2
Non-PHI VPC
AWS Directory
Service
AWS
Device Farm
PHI
Consult with compliance and security organizations before implementing
Indirection Strategy
Validation
/ Edit
System
(EC2)
Inbound
Claim Store
(S3)
HTTPS
Send
SQS
SNS
Claims
PHI Data
Consult with compliance and security organizations before implementing
Example: Simplified Claims Workflow
Validation /
Edit System
(EC2)
Insight
System
(EMR)
Inbound
Claim
Archive
(Glacier)
Inbound
Claim Store
(S3)
Claim
History
(Redshift)
Claims
Adjudication
System
(EC2)
Data
Lake
(S3)
Insights
Consult with compliance and security organizations before implementing
Non-PHI
Insights AWS
LambdaAmazon
SES
Non-PHI
Insights
Email to
Business
Users
SQSSQS
AWS
CodeCommitAWS
CodeDeploy
AWS
CodePipeline
PHI Insights
Non-PHI Insights
Compliance Example Workflow (using DevSecOps)
CloudFormation
templateSecurity /
Compliance Admin
1
Define
AWS Service Catalog
2
Publish
CloudFormation
stack
Healthcare
Developers
4
Browse and Launch
AWS CloudTrail Amazon S3
11
Monitors
Logs all API calls
AWS CloudWatchalarm
8
Monitors
10
Initiates
12
Notifies
AWS Config
Track changes
3
Git push
6
AWS CodeCommit
5
Provisions
9
7
Consult with compliance and security organizations before implementing
Example: Fortune 500 Life Science Company
The Vision
• Self Service
• Rapid Provisioning
• Capacity Management
• Full Stack Availability
Enable Agility
• AD Integration
• Golden AMIs
• Enterprise Logging
• Backup and Retention
• Firewall and Security Rule
Ensure Policy
• Monitoring and Alerts
• VM Scheduling
• Encryption
• Software Configuration Management
Accelerate Best Practices
What they did…
Assurance Monitors
Compliance Database
Console
Billing Roll up
Administrative
Services
Access Control with
AD Integration
User Help
HPC
Workspaces
Big Data
Consult internally before implementing
The following slides are practices we
have seen used in industry. As security
and industry compliance is determined
by the customer before implementing
please:
• Consult with your internal best
practices
• Consult with with your Cloud Center of
Excellence
• Consult with your Information Security
group
• Consult with your Compliance
organization
• Do your due diligence
Thank You
and Healthcare Analytics
Ujjwal Ratan
Healthcare and Life Sciences Solutions Architect
Amazon Web Services
Data
Warehousin
g
Databases
Object and
File Storage
Managed Big
Data Platform
AWS Data Pipeline
Data management ecosystem Analytical tooling ecosystem
Machine
Learning
Analysi
s
Data Ingestion
Storag
e
Archiving
Structured Unstructured StreamingData
Visualization
Typical Analytics Workflow
Retrospective Analysis & Reporting
Amazon S3
Amazon
DynamoDB
Amazon RDS
Ingest Store Process Visualize
Amazon Mobile
Analytics
Amazon
EC2
AWS
Import/Export
Amazon EMR
Amazon Redshift
Amazon
Lambda
Amazon
QuickSight
Three Essential Services for Analytics on AWS
Amazon S3 Amazon
Redshift
Amazon
Elastic
MapReduce
(EMR)
All three are HIPAA eligible services
Store anything
Object storage
Scalable
Designed for 99.999999999%
durability
Amazon S3
Transferring data into Amazon S3
AWS Import/ Export
AWS Direct Connect
Internet
Amazon S3
Data Lake
AWS Region
Institutional Data
Center
Amazon
Analytics
Services
Availability Zone
Aggregate all of your data in Amazon S3 Data
Lake
EMR Kinesis
Redshift DynamoDB RDS
Data Pipeline
Spark StreamingCassandraStorm
Amazon S3
Petabyte scale
Massively parallel
Relational data warehouse
Fully managed; zero admin
Amazon
Redshift
a lot faster
a lot cheaper
a whole lot simpler
When is Amazon Redshift the Right Choice for Healthcare Analytics?
Institutional metrics
Utilize massive datasets with existing SQL skill sets
Queries that involve heavy aggregation such as financial reporting
Clinically actionable gene mutation research
Combine gene variant data with phenotypes and run GWAS/PWAS
analysis using SQL queries
Large population public health studies
Find trends over millions of CMS claims in seconds
Amazon Redshift Architecture
Leader Node
SQL endpoint
Stores metadata
Coordinates query execution
Compute Nodes
Local, columnar storage
Execute queries in parallel
Load, backup, restore via S3
Parallel load from DynamoDB or SSH
HW optimized for data processing DW1: HDD; scale from 2TB to 1.6PB
DW2: SSD; scale from 160GB to 256TB
10 GigE
(HPC)
Ingestion
Backup
Restore
JDBC/ODBC
Copy Data Into Redshift From S3
COPY <table_name> from 's3://<bucket_name>/<file_name>' CREDENTIALS
'aws_access_key_id=<access_key_ID>; aws_secret_access_key=<secret_access_key_id>' DELIMETER ','
IGNOREHEADER 1;
Table_name: Redshift Table Name
Bucket_name: S3 bucket name
File_name: CSV file name in S3 bucket
Access_key_if, secret_access_key_id: AWS security credentials
Hadoop 1.x & 2.x / HDFS clusters
Easy to use; fully managed
Support for EC2 Spot Instances
S3, DynamoDB, Redshift
& Kinesis Integration
Amazon
Elastic
MapReduce
(EMR)
Process – Amazon EMR
• Hadoop - An open-source framework for parallel
processing huge amounts of data on a cluster of
machines
• Amazon EMR - Fully managed Hadoop cluster with
direct integration into Amazon S3 and burstable
capacity
Aggregate the
results from all
nodes and know
what each user did
Process – Amazon EMR Use Case
Large amount of
click logs of user
actions in Amazon
S3 bucket
(e.g TBs)
Amazon EMR cluster
splitting logs into
small pieces working
in parallel
Process – Amazon EMR
• Amazon EMR supports all common Hadoop Frameworks
such as:
• Spark, Pig, Presto, Hive
• etc.
• Decouples storage from compute
• Allows independent scaling
• Direct Integration with DynamoDB and S3
Amazon S3Amazon
DynamoDB
Amazon EMR
Amazon EMR + Hue
S3, Redshift & EMR forms the backbone of most
analytical workflows on AWS.
When used with other AWS services,
this is how the final architecture would look like …......
EC2
Amazon EC2
Instances
Amazon
Kinesis
Amazon S3
Amazon
EMR
Amazon
Redshift
BI Tool
Amazon
Machine
Learning
Amazon
DynamoDB
Amazon Mobile
Analytics
Amazon
Lambda
AWS
Import/Export
Security and Compliance
Visibility for HealthcareAWS Nashville Event – May 2016
Adam C. GreenfieldDirector of Engineering
HEALTHCARE Exclusive
CLOUD
Experts
CERTIFIED
Experience
• BAA with the most coverage of any
leading provider
• Incorporates existing infrastructure
BAAs into a single BAA
THE CLEARDATA DIFFERENCE
ENHANCED
BAA
Deployment Tools
• Configuration Management Tools
• Orchestration Tools
• Auditing and Governance Tools
57PROPRIETARY & CONFIDENTIAL
58PROPRIETARY & CONFIDENTIAL
59PROPRIETARY & CONFIDENTIAL
Objectives
Strong and
Secure Audit
Trail
No tight
coupling to
orchestration
tools
External
Managed
Services
Highly
Automated
60PROPRIETARY & CONFIDENTIAL
Traditional Platforms
• Platforms normally sit between your
application and tools to translated API
calls into AWS functions.
• This creates vendor lock in, but
obscures AWS value and reduces agility
• Vendors must integrate new services
quickly to give customers access to AWS
features
Customer Applications & Tools
Vendor Platform & Custom API’s
DB on instanceinstance with AMI
Rethinking the model
• Observe
• Orient
• Decide
• Act
62PROPRIETARY & CONFIDENTIAL
Objectives
Credits: Patrick Edwin Moran https://commons.wikimedia.org/wiki/File:OODA.Boyd.svg
63PROPRIETARY & CONFIDENTIAL
AWS ConfigAWS CloudTrail
AWS CloudWatch
Customer Account
AWS SNSAmazon API
Gateway
Management Account
AWS
Lambda
Amazon
Kinesis
64PROPRIETARY & CONFIDENTIAL
Kinesis Streams
SensuCMDB
Backups Vuln Scanning
SlackPagerDuty
Ticketing
CloudTrail / CloudWatch Events EC2 Events Auditing / Governance
AlertingSEIM
Remediation
Amazon
DynamoDB
Amazon
Redshift
Configuration with tags
66PROPRIETARY & CONFIDENTIAL
Trusted Advisor
• Catches common account misconfigurations
• Suggests cost reductions
• Evaluates fault tolerance
67PROPRIETARY & CONFIDENTIAL
CloudWatch
• Monitor performance of AWS resources
• Aggregate and process log files (non-PHI)
• Requires instance profile or distributed credentials
68PROPRIETARY & CONFIDENTIAL
Emerging AWS-native Solutions
AWS Config Rules
https://github.com/awslabs/aws-config-rules/
Community-Based Rules • Constantly watch for account changes
• Remediate in near real-time
• Incredibly flexible and extendable
• Lambda based
69PROPRIETARY & CONFIDENTIAL
Emerging AWS-native Solutions
Extending OODA inside the instance
• Observe
• Orient
• Decide
• Act
71PROPRIETARY & CONFIDENTIAL
Objectives
Strong and
Secure Audit
Trail
Unobtrusive
External
Managed
Services
Highly
Automated
72PROPRIETARY & CONFIDENTIAL
ClearDATA Dynamic Cloud Platform
AWS Environment
• Compute
• Storage
• Network / Cloud
Operating Environment
• Hardened AMIs
• Configuration management engine
• Patch management
• Managed backup
• Monitoring & alerts
• Consolidated account info
• Isolated dev & test environments
Security & Compliance
• Hardened encryption configuration
• Key management
• Intrusion detection system
• Login and access tracking
• Event log management
• File integrity monitoring
• ClearDATA security appliance
• VPNs / Address translation
• Anti-virus
24/7 Managed Services
Delivered by AWS Certified Personnel
Over 30 additional services automatically attached to AWS infrastructure
73PROPRIETARY & CONFIDENTIAL
• First of it’s kind in the
industry – service based
real-time HIPAA compliance
dashboard
• At a glance system status
plus trending over time
• Detailed history available for
attestation during audits
Continuous security and compliance
monitoring mapped directly to
HIPAA guidelines delivered across
cloud and private environments via
interactive dashboard and individual
asset scorecards.
Security & Compliance Dashboard
74PROPRIETARY & CONFIDENTIAL
Cloud Platform BAA Coverage
AWS Global
Infrastructure
Availability Zones
Regions
Edge
Locations
AWS Foundation Services
Compute Storage Database Networking
Network Traffic
Protection
Server-Side
Encryption
Client-Side Data
Encryption
Operating Systems, Network & Firewall Configurations
Platform
Customer Data
Applications Identity & Access Management
AWS Global
Infrastructure
Availability Zones
Regions
Edge
Locations
AWS Foundation Services
Compute Storage Database Networking
Network Traffic
Protection
Server-Side
Encryption
Client-Side Data
Encryption
Operating Systems, Network & Firewall Configurations
Customer Data
ClearDATA
PlatformApplications Identity & Access Management
Amazon Web Services Infrastructure ClearDATA Cloud Platform
HEALTHCARE
Exclusive
CLOUD
Experts
CERTIFIED
Experience
• Current Projects
• Pilots or POCs
• Backup / DR
• Compliance Dashboard
• SRA / SRAaaS
• Cloud Assessment
THANK YOU!
ENHANCED
BAA
LET’S WORK
TOGETHER
Data Storage for the Long Haul
Compliance and Archive
Erik Durand
Amazon Web Services
Amazon EFS
File
Amazon EBSAmazon EC2
Instance Store
Block
Amazon S3 Amazon Glacier
Object
Data Transfer
AWS Direct
Connect
AWS
Snowball
ISV Connectors Amazon
Kinesis
Firehose
S3 Transfer
Acceleration
Storage
Gateway
Storage is a platform
Patient data – Philips Healthcare
• HealthSuite digital platform powered by AWS
• 15 petabytes of patient data
• Archived for decades (beyond the lifetime of patients)
• Uses AWS HIPAA eligible services in the BAA
Public sector – King County
• Most populous county in Washington state
• Replace tape solution for backup from 17 agencies
• Meet compliance requirement
• Saved $1MM in first year, no more tape refresh or
management churn
Archive:
Data retained for the long term,
for compliance or potential
future reference
Data archiving needs are growing everywhere
• Media assets, 4K, 8K
• Health care / life sciences
• Financial services
• Regulated industries
• Oil and gas / geospatial
• Digital preservation
• Long-term backups
• Logs
Traditional archiving approaches
• Storage arrays / disk arrays
• Tape silos / tape libraries
• Tape drives (LTO-X / DLT / etc.)
• Virtual tape libraries (VTLs)
• Tape out / vaulting
• Specialized software and personnel
How can AWS help with your archival?
Metered usage:
Pay as you go
No capital investment
No commitment
No risky capacity planning
Avoid risks of physical
media handling
Control your
geographic locality for
performance and
compliance
Archive Options – Storage Tiers and Data Lifecycle
Object Storage Options
S3 Standard
Active data Archive dataInfrequently accessed data
S3 Standard - Infrequent
Access
Amazon Glacier
Milliseconds 3-5 hoursMilliseconds
$0.03/GB/mo $0.007/GB/mo$0.0125/GB/mo
A Closer Look: S3-IA and Amazon Glacier
S3 - IA
• Same durability and throughput as S3 Standard
• Instant access
• $0.01/GB on each data retrieval
Amazon Glacier
• Same 11 9s durability as S3 Standard
• 3-5 hour data retrieval latency
• Suitable for cold archive such as offsite tapes
S3 Standard - Infrequent
Access
Amazon Glacier
- Transition Standard to Standard-IA
- Transition Standard-IA to Amazon Glacier
- Expiration lifecycle policy
- Versioning support
Data lifecycle management
T T+3 days T+5 days T+ 15 days T + 25 days T + 30 days T + 60 days T + 90 days T + 150 days T + 250 days T + 365 days
Data access frequency over time
Setup lifecycle policy
Transition older records to Standard-IA
Archive to S3-IA after 30 days
Lifecycle policy
Standard Storage -> Standard-IA
<LifecycleConfiguration>
<Rule>
<ID>sample-rule</ID>
<Prefix>documents/</Prefix>
<Status>Enabled</Status>
<Transition>
<Days>30</Days>
<StorageClass>STANDARD-IA</StorageClass>
</Transition>
<Transition>
<Days>365</Days>
<StorageClass>GLACIER</StorageClass>
</Transition>
</Rule>
</LifecycleConfiguration>
Archive to Amazon Glacier after 365 days
Lifecycle policy
Standard Storage -> Standard-IA
<LifecycleConfiguration>
<Rule>
<ID>sample-rule</ID>
<Prefix>documents/</Prefix>
<Status>Enabled</Status>
<Transition>
<Days>30</Days>
<StorageClass>STANDARD-IA</StorageClass>
</Transition>
<Transition>
<Days>365</Days>
<StorageClass>GLACIER</StorageClass>
</Transition>
</Rule>
</LifecycleConfiguration>
Standard-IA Storage -> Amazon Glacier
Save money on storage
58% saving over S3 Standard
44% saving over S3 Standard-IA
* Assumes the highest public pricing tier
Example backup software integration
• CommVault – Native Integration
with Amazon S3 and
Amazon Glacier
• Deduplication and encryption
• Single console management
Amazon S3 Amazon Glacier
Compliance Use Case 1 – Regulatory Retention
Amazon Glacier Vault Lock allows you to easily
set compliance controls on individual vaults and enforce them via a
lockable policy
Time-based retention
MFA authentication
Controls govern all
records in a Vault
Immutable policy
Two-step locking
Compliance storage with Vault Lock
Vault Lock for compliance storage
• Non-overwrite, non-erasable records
• Time-based retention with “ArchiveAgeInDays” control
• Policy lockdown (strong governance)
• Legal hold with vault-level tags
• Configure optional designated third-party access and grant
temporary access
Amazon Glacier received a third-party assessment
from Cohasset Associates on how Amazon Glacier
with Vault Lock can be used to meet the requirements
of SEC Rule 17a-4(f) and CFTC 1.31(b)-(c).
Example control: 1 year record retention
• Deny delete archive operation
• From anybody (root, administrators, users, business partners)
• When ArchiveAgeInDays is <= 365 days
Archive age computed from the time an archive lands in a vault
Example control: 1 year record retention
Vault Lock: Two-step locking
• InitiateVaultLock
– Effectuates a retention policy for testing (in-progress state)
– Returns a unique lock ID (expires after 24 hours)
• AbortVaultLock
– Deletes an in-progress policy
– Ability to modify a policy before locking it down
• CompleteVaultLock
– Locks down the vault with the appropriate lock ID
– Vault Lock cannot be aborted afterwards
Legal hold with vault-level tags
• Set up a legal hold tag
– Configure a vault-level tag “LegalHold”
– Set initial value to “False”
• Add compliance control for legal hold in a Vault Lock policy
– Deny delete archive operation
– From anybody (root, administrators, users, business partners)
– When LegalHold tag = “True”
• Place/lift legal hold by updating the tag value
Example control: Legal hold
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Vault Lock in the Amazon Glacier console
Compliance Use Case 2 – Auditing and Alerts
Audit logging with AWS CloudTrail
• Amaozn S3 and Amazon Glacier can log
API calls for audit via CloudTrail
• Enable CloudTrail in the AWS console and
designate your log bucket
• S3 logs bucket-level activities; object
activities supported via event notification
• Amazon Glacier logs all APIs calls for
vault and archives
Access policy for a storage container
• Control access to a storage container in a single location
– S3 bucket or Amazon Glacier vault access policy
– Grant/revoke access to internal business units/teams
– “Marketing_Vault” has a distinct access policy from “DevOps_Vault”
• Easily manage cross-account access for your business partner
– Simply add a section for your business partner in the same policy
– Cross-account activities (API calls) also show up in CloudTrail logs
Amazon S3 event notifications
Events
SNS topic
SQS
queue
Lambda
function
• Notification when objects are
created via PUT, POST, Copy, or
Multipart Upload, DELETE
• Filtering on prefixes and suffixes
for all types of notifications
Request specific notifications
Request notifications on specific
PUT APIs
Request notifications on specific
DELETE APIs
s3:ObjectCreated:*
s3:ObjectCreated:Put
s3:ObjectCreated:Post
s3:ObjectCreated:Copy
s3:ObjectCreated:CompleteMultipartUpload
s3:ObjectRemoved:*
s3:ObjectRemoved:Delete
s3:ObjectRemoved:DeleteMarkerCreated
Compliance Use Case 3 – Geographic Redundancy
Remote replicas managed
by separate AWS accounts
Secure
Distribute data to regional
customers
Lower Latency
Store hundreds of
miles apart
Compliance
Amazon S3 cross-region replicationAutomated, fast, and reliable asynchronous replication of data across AWS regions
• Usual charges for
storage, requests, and
inter-region data transfer
for the replicated copy of
data
• Replicate into Standard-IA
or Amazon Glacier
Cost
HEAD operation on a source
object to determine replication
status
• Replicated objects will not be
re-replicated
• Use Amazon S3 COPY to
replicate existing objects
Replication status
DELETE without object
version ID• Marker replicated
DELETE specific object
version ID• Marker NOT replicated
Delete operation
Cross-region replication: Details
Object ACL updates are
replicated
• Objects with Amazon-
managed encryption key
replicated
• AWS KMS encryption not
replicated
Access control
Versioning with cross-region replication
A
B
Vid1- v2
Vid1- v1
Key: A/vid1 Key: B/vid1
Vid1- v2
Vid1- v1
Vid1- v3Vid1- v3
Vid1- v4Vid1- v4
A
Cross-region replication with lifecycle archiving
S3
Bucket A
Amazon Glacier
S3
Bucket B
AWS Import/Export Snowball
• Accelerate PBs with AWS-
provided appliances
• 80 TB model, global availability
AWS Storage Gateway
• Instant hybrid cloud
• Up to 120 MB/s cloud upload rate
(4x improvement), and
Data ingestion into AWS storage services
Amazon Kinesis Firehose
• Ingest data streams directly into
AWS data stores
AWS Direct Connect
• COLO to AWS
ISV Connectors
• CommVault
• VERITAS
• etcetera
Amazon S3 Transfer Acceleration
• Move data up to 300% faster
using AWS’s private network
What is AWS Snowball? Petabyte scale data transport
E-ink shipping
label
Ruggedized
case
“8.5G Impact”
All data encrypted
end-to-end50TB or 80 TB
10G network
Rain & dust
resistant
Tamper-resistant
case & electronics
How it works
Introducing Amazon S3 transfer acceleration
S3 BucketAWS Edge
Location
Uploader
Optimized
Throughput!
Typically 50%–400% faster
Change your endpoint, not your code
54 global edge locations
No firewall exceptions
No client software required
Amazon
Route 53
Resolve
b1.s3-accelerate.amazonaws.com
HTTPS PUT/POST
upload_files.zip
HTTP/S PUT/POST
“upload_files.zip”
Service traffic flowClient to S3 Bucket example
S3 Bucket
b1.s3-accelerate.amazonaws.com
EC2 Proxy
AWS Region
AWS Edge Location
Customer Client
1
2
3
4
AWS Snowball S3 transfer acceleration
When do I use what?
Large, infrequent uploads
Tens of TBs of upload from a
centralized location
7–10 day tolerance
Recurring, frequent uploads
GBs or TBs of upload from distributed
locations
Long geographic distances
Q&A
Learn more at: http://aws.amazon.com/s3/
http://aws.amazon.com/glacier/
http://aws.amazon.com/importexport/