Date post: | 15-Apr-2017 |
Category: |
Software |
Upload: | freeman-zhang |
View: | 170 times |
Download: | 0 times |
AMAZON WEB SERVICESSHIJIE ZHANG
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
BACKGROUND-CLOUD
WHY PUBLIC CLOUD?
Cost savings Scalabilty Flexibility Training
Pay only for operational costs vs ownership
costs
Rapid expansion local & global
Change hardware configurations Set up lab instantly
Pay only for usage vs over provisioning Disaster recovery Adapt hardware to
baseline dynamically Try new features
BACKGROUND-CLOUD
CLOUD USAGE MODEL - HYBRID CLOUDS
AWS
Azure
IBMVMware
FujitsuSunGard
Rackspace
ATT VerizonQuest
Unisys
your own servers
CSC
Hyperscale Cloud ProvidersNormal Cloud Providers
Public Cloud
Private Cloud
BACKGROUND-CLOUD
CLOUD USAGE MODEL - HYBRID CLOUDS
AWS
Azure
IBMVMware
FujitsuSunGard
Rackspace
ATT VerizonQuest
Unisys
your own servers
CSC
Hyperscale Cloud ProvidersNormal Cloud Providers
Public Cloud
Private Cloud
2009
2%
98%
BACKGROUND-CLOUD
CLOUD USAGE MODEL - HYBRID CLOUDS
AWS
Azure
IBMVMware
FujitsuSunGard
Rackspace
ATT VerizonQuest
Unisys
your own servers
CSC
Hyperscale Cloud ProvidersNormal Cloud Providers
Public Cloud
Private Cloud
2014
6%
94%
BACKGROUND-CLOUD
CLOUD USAGE MODEL - HYBRID CLOUDS
AWS
Azure
IBMVMware
FujitsuSunGard
Rackspace
ATT VerizonQuest
Unisys
your own servers
CSC
Hyperscale Cloud ProvidersNormal Cloud Providers
Public Cloud
Private Cloud
2099?
~90%
~10%
BACKGROUND-CLOUD
DEVELOPER’S ROLE IN CLOUD
▸ Sounds like IT Pros’ problems rather than developers’? No
▸ Costs / Security / Integration / …
▸ Developer one of main target users
▸ Code applications aware of infrastructure (Assume infrastructure can fail e.g. Netflix)
▸ Host dev env / Deploy web app / Leverage cloud database…
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
BACKGROUND-AWS
BACKGROUND-AWS
AWS FREE TIER
▸ Great for getting a feeling
▸ Free for the first 12 months
BACKGROUND-AWS
AWS SUPPORT▸ Comprehensive documentation (Console / Native API / SDK)
▸ Tiers of support (developer / business / enterprise )
MAJOR USE CASES
BROAD AND DEEP INFRASTRUCTURE AS A SERVICE
▸ Give a detailed example later
MAJOR USE CASES
RICH PLATFORM SERVICES
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Backgrounds
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Backgrounds
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
SAMPLE APPLICATION
SAMPLE APPLICATION
SAMPLE APPLICATION
OVERALL ARCHITECTURE
AWS GLOBAL INFRASTRUCTURE
11 Regions & 30 Availability Zones - December 2015
5 More Regions & 10 More Availability Zones
A DETAILED EXAMPLE
REGIONS AND ZONES
▸ Region: a geographical area
▸ Availability Zone: a data center
▸ Different regions may have different services/prices
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
AWS COMPUTE SERVICES
▸ EC2
▸ Auto Scaling
▸ Lambda
▸ EC2 Container Service (for integration with docker)
▸ EMR (Amazon’s Hadoop implementation)
A DETAILED EXAMPLE
ELASTIC COMPUTER CLOUD ( EC2 )
▸ Def: Instance provisioning and shutting down service
▸ AMI def: Amazon Machine Image, virtual disk template (OVA, OVF)
▸ AMI instance types: (Support Windows / Linux)
▸ AMI pricing types:
Micro instances
General purpose
Compute optimized
GPU instances
Memory optimized
Storage optimized
On-demand Instances Reserved Instances Spot Instances
A DETAILED EXAMPLE
AUTO SCALING
▸ Def: expand or shrink EC2 instances on demand
▸ Triggers: Manual schedule or integrated with monitoring
load balancer
dns
A DETAILED EXAMPLE
AUTO SCALING
▸ Def: expand or shrink EC2 instances on demand
▸ Triggers: Manual schedule or integrated with monitoring (CloudWatch)
load balancer
dns
A DETAILED EXAMPLE
LAMBDA
▸ Def: Event-Driven compute service
▸ Does not require an instance, simplifying response to events
▸ Type of events: (Any API call or resource transition)
▸ Put objects in S3
▸ Transition in an EC2 instance
▸ Write to a database table
▸ Use cases:
▸ Generate thumbnail images as arriving in S3
▸ …
COMPUTE WITHIN ARCHITECTURE
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront, Glacier)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
STORAGE SERVICES
▸ Ephemeral storage
▸ EBS
▸ S3 ( simple storage service)
▸ Glacier
▸ CloudFront
} Block Storage
Access through OS at device level
} Object Storage
Access through HTTP at user level
} Specialized purpose storage
A DETAILED EXAMPLE
Definition Durability Accessibility IOPS Snapshot
Storage coming with EC2 instance
Just C drive coming with PC
lost once instance terminated (not reboot)
Locked to one instance
No performance guarantee
Support with instance
Elastic block storage
Just like external disk for PC
persist until deleted
independently
Can attached to multiple instances
one at a timeSLA Support
independently
BLOCK STORAGE
A DETAILED EXAMPLE
SIMPLE STORAGE SERVICE (S3)
▸ History: First service by AWS
▸ Def: An storage bucket for objects
▸ Size: Unlimited bucket size, Up to 5TB object size
▸ Accessibility: HTTP/HTTPS
▸ Not: a file system (vs Dropbox)
A DETAILED EXAMPLE
SIMPLE STORAGE SERVICE (S3)
▸ Types:
▸ Standard storage:
▸ 99.999,999,999% durability
▸ $0.03 per GB / month
▸ Use cases: Master storage
▸ Reduced redundancy storage:
▸ 99.99% durability
▸ $0.024 per GB / month
▸ Use cases: Slave storage
A DETAILED EXAMPLE
GLACIER
▸ Price: Very cheap, $0.007 per GB / month (Region: Virginia)
▸ Usage: Ideal for backup
▸ Retrieval time: Very, very slow (4-6 hours)
A DETAILED EXAMPLE
CLOUDFRONT
▸ Definition: Global content delivery network service
▸ Infrastructure behind: Edge locations (CDN endpoints for CloudFront)
▸ Number: Over 50 Edge Locations, many more than regions
Blue: edge locations
Yellow: regions
STORAGE WITHIN ARCHITECTURE
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
DATABASE SERVICES
▸ RDS
▸ DynamoDB
▸ ElastiCache
▸ Redshift
A DETAILED EXAMPLE
RDS
▸ Def: Provision database instances
▸ Engines:
▸ Storage options:
MySQL PostgreSQL Oracle Microsoft SQL Server
Amazon Aurora
Storage Size IOPS Price
General purpose SSD 5GB — 3TB 3 IOPS per GB storage only
Elastic block storage 100GB — 3TB 1,000 ~ 30,000 IOPS per GB storage + IOPS
Magnetic storage 5GB — 3TB ~100 IOPS per GB Storage + I/O rate
A DETAILED EXAMPLE
RDS
▸ Hardware acquisition and upkeep (spacing, cooling)
▸ OS configuration and maintenance
▸ Database installation
▸ Database configuration
▸ Database patch
▸ Database backup
▸ Database scaling (give you powerful tools)
What aspects are managed?
What are left over?▸ Design schema
▸ Optimizing
▸ Scaling (with baked-in powerful tools)
A DETAILED EXAMPLE
RDS — SCALE UP▸ Increase storage amount
▸ Increase storage type (SSD, Magnetic, ..)
▸ Change instance class (CPU, RAM)
A DETAILED EXAMPLE
RDS — SCALE OUT▸ Read replica: DB replica for read access
▸ Use case: read-intensive applications / source for reports or analytics
▸ Can add additional indexes
▸ At least be same size as source DB instance
▸ Multi-AZ deployment: Synchronous standby (not eventually) in different AZ
▸ Use cases: Reduce latency during maintenance, automatic failover
A DETAILED EXAMPLE
DYNAMODB▸ Key-based noSQL DB
▸ When creating table, just need define primary keys, not schema
▸ Additional replica (happens behind scenes)
▸ Scaling up / out (happens behind scenes)
A DETAILED EXAMPLE
ELASTICACHE▸ Def: Cluster tied to single AZ,Distributed in-memory cache service
▸ Implemented based on Memcached
▸ Managed (Automatically fix failure nodes)
▸ Scale out (Add/Remove nodes)
A DETAILED EXAMPLE
ELASTICACHE
DATABASE WITHIN ARCHITECTURE
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
NETWORKING
▸ Virtual Private Cloud
▸ Elastic Load Balance
▸ Route53
A DETAILED EXAMPLE
VIRTUAL PRIVATE CLOUD▸ Usage: for building data center in AWS cloud
A DETAILED EXAMPLE
VIRTUAL PRIVATE CLOUD▸ VPC Peering
A DETAILED EXAMPLE
VIRTUAL PRIVATE CLOUD▸ AWS VPN
A DETAILED EXAMPLE
ELASTIC LOAD BALANCE▸ Usage: Distribute traffic across EC2 instances in one AZ, or multiple
▸ Distributed and fault tolerant built in
A DETAILED EXAMPLE
ROUTE 53▸ Def: DNS web service from AWS
▸ Usage: Map names to IP addresses / Load balancing between regions
▸ Not a domain registrar
▸ Routing targets: Route to CloudFront / ELB / websites running in S3
NETWORKING WITHIN ARCHITECTURE
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
MESSAGING
▸ SNS ( Simple Notification Service )
▸ SQS ( Simple Queue Service)
A DETAILED EXAMPLE
SIMPLE QUEUE SERVICE (SQS)
▸ Def: Highly available, scalable queue storage
▸ Usage:flow control / buffer / decoupling apps
▸ Size: Queue unlimited in queue size, single message up to 256 kb in size
▸ Costs: $0.50 / million SQS requests
A DETAILED EXAMPLE
SIMPLE NOTIFICATION SERVICE (SNS)
▸ Def: High available, scalable message broadcasting service
▸ Subscript to a topic, subscribers notified by HTTP/HTTPS/SMTP/SMS/SQS when new messages are available
▸ Usage: time-sensitive info updates, mobile app updates
MESSAGING WITHIN ARCHITECTURE
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
SECURITY
▸ IAM (Identity Access and Management)
▸ Instance security
A DETAILED EXAMPLE
IAM
▸ Def: Hub for control AWS permissions
▸ Role / Group / User — Policy
A DETAILED EXAMPLE
INSTANCE SECURITY
▸ Subnet layer - NACL (like firewalls for subnets)
▸ Instance layer - Security group (like firewall for instances)
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
MONITORING
▸ CloudWatch - AWS service metrics
▸ CloudTrail - all AWS API call logs
A DETAILED EXAMPLE
CLOUDWATCH
▸ Usage:
▸ Provides metrics,
▸ Raise alarm (send notification, stop servers, lambda)
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
A DETAILED EXAMPLE
DEPLOYING
▸ CloudFormation
▸ Elastic Beanstalk
A DETAILED EXAMPLE
CLOUDFORMATION / BEANSTALK
▸ Def: Automate stack of AWS resource provisioning / deleting
▸ Comparison: Similar purpose. Later one more for developers (Infrastructure as code in different sdks)
OUTLINE
▸ Background
▸ Cloud
▸ AWS
▸ A Detailed Example
▸ Background
▸ Compute (EC2, Auto Scaling, Lambda)
▸ Storage and Content Delivery (EBS, S3, CloudFront)
▸ Database (RDS, Dynamo, ElastiCache)
▸ Networking (ELB, Route 53, Virtual Private Cloud)
▸ Messaging (SQS, SNS)
▸ Security (IAM)
▸ Monitoring (CloudWatch, CloudTrail)
▸ Deploying (Beanstalk, CloudFormation)
▸ Summary
OVERALL ARCHITECTURE
SUMMARY
▸ Many cloud services are high scalable by default (S3, ELB, SQS, …). Some is ready for high scalability, but needs a little more work (RDS, EC2)
▸ AWS supports accessing service via GUI, SDK, Native APIs
▸ Utilizing AWS cloud in developing
REFERENCES
▸ https://app.pluralsight.com/library/courses/deploying-highly-available-distributed-systems-aws-part1/table-of-contents
▸ https://app.pluralsight.com/library/courses/deploying-highly-available-distributed-systems-aws-part2/table-of-contents
▸ https://app.pluralsight.com/library/courses/amazon-web-services-databases-in-depth/table-of-contents
▸ https://app.pluralsight.com/library/courses/aws-vpc-operations/table-of-contents
▸ https://app.pluralsight.com/library/courses/aws-course/table-of-contents
▸ https://app.pluralsight.com/library/courses/aws-system-admin-fundamentals/table-of-contents
▸ https://acloud.guru/courses
THANKS