Date post: | 14-Jan-2015 |
Category: |
Technology |
Upload: | amazon-web-services |
View: | 1,505 times |
Download: | 3 times |
Cloud Best Practices Whitepaper Prescriptive guidance to Cloud Architects
http://bit.ly/aws-best-practices
Abstract Resources
No Servers or Hard drives but Instances and Volumes. Cloud resources are fungible.
On-Demand Provisioning
Ask for what you need, exactly when you need it. Get rid of it when you don’t need.
Scalability in Minutes
Scale out or in depending on usage needs.
Pay On Consumption
You stop paying for resources when you turn them off
Automation Cloud gives you access to scriptable infrastructure. Allows you to automate using APIs.
Cloud Computing Attributes Why Architects love the cloud?
The “Living and Evolving” Cloud The “Living and Evolving” Cloud AWS services and basic terminology
Infrastructure building blocks
Platform building blocks
Tools to access services
Cross Service features
The “Living and Evolving” Cloud AWS Building Blocks
Inherently Fault-Tolerant
Services
Fault-Tolerant
with the right
architecture Amazon S3
Amazon
DynamoDB
Amazon
CloudFront
Amazon SWF
Amazon SQS
Amazon SNS
Amazon SES
Amazon Route53
Elastic Load
Balancing
AWS IAM
AWS Elastic
Beanstalk
Amazon
ElastiCache
Amazon EMR
Amazon
CloudSearch
Amazon
Redshift
Amazon EC2
Amazon EBS
Amazon RDS
Amazon VPC
Scalability
Characteristics of Truly Scalable Service
Build a Scalable Architecture on AWS
A scalable architecture is critical to take advantage of a scalable infrastructure
Increasing resources results in a proportional increase in performance
A scalable service is operationally efficient
A scalable service is resilient
A scalable service becomes more cost effective when it grows
Cloud Architecture Lessons
1. Design for failure and nothing fails 2. Loose coupling sets you free 3. Implement “Elasticity” 4. Build Security in every layer 5. Think Parallel 6. Leverage different storage options
using Amazon Web Services
1. Design for Failure
"Everything fails, all the time" Werner Vogels, CTO Amazon.com
and nothing will really fail
Avoid single points of failure Assume everything fails, and design backwards Goal: Applications should continue to function even if the underlying physical hardware fails or is removed or replaced.
Design for Failure with AWS Tools to make your life easier
Use Fault-tolerant Services as Ingredients of your App Use Amazon Elastic Block Store (EBS) Snapshots Auto-scaling for Auto-Recovery Multi-AZ Data Replication and Recovery On-demand application provisioning in a different AZ Multi-AZ Application Deployment and Data replication
2. Build Loosely Coupled Systems The looser they're coupled, the bigger they scale
Independent components Design everything as a Black Box De-couple for Hybrid models Load-balance clusters
Controller A Controller B Controller C
Controller A Controller B Controller C
Q Q Q
Tight Coupling
Loose Coupling using Queues
Use Amazon SQS as Buffers
3. Implement Elasticity
Don’t assume health or fixed location of components Use designs that are resilient to reboot and re-launch Bootstrap your instances: Instances on boot will ask a question “Who am I & what is my role?” Enable dynamic configuration
Elasticity is fundamental property of the Cloud
Use Auto Scaling Use Elastic Load Balancing on multiple layers Use configurations in DynamoDB to bootstrap instance Use Configuration Management tools like Chef & Puppet…
3. Implement Elasticity Towards elastic architectures
Resilient to reboot and re-launch: Design the system such that in the event of a failure, it is resilient enough to automatically re-launch and restart. Forcefully fail and test (Chaos Monkey). Stateless: Extract stateful components and strive to make them stateless. Packable into an AMI: Package and deploy your application into an AMI so it can run on an Amazon EC2 instance. Run multiple instances on multiple Amazon EC2 instances. Decouple: Isolate the components using Amazon SQS. Decouple code with deployment and configuration.
Standardized Technology Stacks Use a Chaos Monkey Standardized Application Stacks
4. Build Security in every layer
In the Cloud, Security is a Shared Responsibility and it has to be implemented in every layer
Design with Security in mind
SOC 1, 2, 3 ISO 27001/2 Certification PCI DSS 2.0 Level 1-5 HIPAA/SOX Compliance FedRAMP FISMA & DIACAP ITAR
Enforce IAM policies Use MFA, VPC, Leverage S3
bucket policies, EC2 Security groups, EFS in EC2 Etc..
Encrypt data in transit Encrypt data at rest
Protect your AWS Credentials Rotate your keys
Secure your application, OS, Stack and AMIs
In the cloud, Security is a Shared Responsibility
Application Security
Services Security
Infrastructure Security
How we secure our infrastructure
What security options and features are available to you?
How can you secure your application and what is your responsibility?
Availability Zone #2
RDS Slave
Availability Zone #1
Cache Tier
Auto Scaling group : Web Tier
RDS Master
Elastic Load
Balancer
Memcache
LB
Tomcat Memcache
Distribution
Buckets
Amazon
CloudFront
Amazon S3
www.myphpwebsite.com
(dynamic data)
media.myphpwebsite.com
(static data)
Amazon Route 53
(DNS)
Amazon EC2
# Permit HTTP(S) access to Web
Layer from the Entire Internet
ec2auth Web -p 80,443 -s 0.0.0.0/0
# Permit Web Layer access to App
Layer
ec2auth App -p 8000 –o Web
# Permit App Layer access to DB
ec2auth DB -p 3209 –o App
# Permit admin access SSH to all
three layers
# First allow connection from
office to Web tier, and from there
to the other layers
ec2auth Web -p 22 -s <for example,
network block of your office>
ec2auth App -p 22 -o Web
ec2auth DB -p 22 -o Web
Web Server
App Server
Web Server
App Server
5. Think Parallel
Experiment with different architectures in parallel Multi-threading and Concurrent requests to cloud services Run parallel MapReduce Jobs on Amazon Elastic MapReduce Use Elastic Load Balancing to distribute load across multiple servers Decompose a Job into its simplest form
Serial and Sequential is now history
Hadoop Amazon Elastic
MapReduce
Cluster Compute
HPC
Spot Instances
Expand/ Shrink
Elastic Super
computer
Distributed Processing Framework
On-demand Infrastructure (Cloud) + Automation
Each VM = 2 Xeon “Nehalem” Quad-core 10G Ethernet 2 GPGPUs
Cost savings due to lower “Spot price” (Time-insensitive tasks)
Expand or Shrink a running cluster
Big Data power house
5. Think Parallel This is where AWS really shines..
6. Leverage many storage options
Amazon S3: large static objects Amazon Cloudfront: content distribution Amazon DynamoDB : data indexing/querying Amazon EC2 local disc drive : transient data Amazon EBS: persistent storage for any RDBMS + Snapshots on S3 Amazon RDS: RDBMS service - Automated and Managed Amazon Redshift: Data warehouse service
Use Scalable Ingredients
6. Leverage many storage options Which storage option to use when?
Amazon S3 + CloudFront
Amazon EC2
Ephemeral Store
Amazon EBS Amazon DynamoDB
Amazon RDS Amazon Redshift
Ideal for Storing Large write-once, read-many types of objects, Static Content Distribution
Storing non-persistent transient updates
Off-instance persistent storage for any kind of data
Querying light-weight attribute data (SimpleDB)
Highly scalable applications (DynamoDB)
Storing and querying structured Relational and referential Data
Storing and rapidly querying vast amounts of Relational data
Ideal examples Media files, audio, video, images, Backups, archives, versioning
Config Data, scratch files, TempDB
Clusters, boot data, Log or data of commercial RDBMS like Oracle, DB2
Querying, Mapping, tagging, click-stream logs, metadata, shared-state management, indexing
Complex transactional systems, inventory management and order fulfillment systems
Enterprise data warehouse,
DW cube lookups
Not recommended for
Querying, Searching
Storing Database logs or backups, customer data
Relational (joins) query
Moderately sized data sets
Not recommended examples
Database, File Systems
Sensitive data
Content Distribution
OLTP, DW cube rollups
Simple lookups
Simple lookups
Cloud Architecture Lessons Best Practices
1. Design for failure and nothing fails 2. Loose coupling sets you free 3. Implement Elasticity 4. Build Security in every layer 5. Think Parallel 6. Leverage many storage options
P h o t o : G r a n d C a n y o n H o p i P o i n t S u n S e t
Additional Info..
AWS Architecture Center - http://aws.amazon.com/architecture AWS Premium Support - http://aws.amazon.com/premiumsupport AWS Blog – http://aws.amazon.com/blog
http://aws.amazon.com