© 2016, Amazon Web Serv ices, Inc. or its Af f iliates. All rights reserv ed.
Konstantin Wilms, Solutions Architect
June 21st, 2016
Amazon EC2 Container
Service Deep Dive
Agenda
Infrastructure Setup
Infrastructure Management
PaaS on ECS
Amazon ECS Infrastructure
Setup
Amazon ECS Cluster Setup
Cluster Setup with AWS CloudFormation
CloudFormation supports ECS cluster, service and task
definition resources
Use AWS::IAM::Role to create ECS service role and
container instances role
Launch container instances using
AWS:AutoScaling::LaunchConfiguation and
AWS:AutoScaling::AutoScalingGroup
Cluster Setup with AWS CloudFormation
"Resources" : {
"ECSCluster": {
"Type": "AWS::ECS::Cluster"
},
"ECSAutoScalingGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"VPCZoneIdentifier" : { "Ref" : "SubnetID" },
"LaunchConfigurationName" : { "Ref" : "ContainerInstances" },
"MinSize" : "1",
"MaxSize" : { "Ref" : "MaxSize" },
"DesiredCapacity" : { "Ref" : "DesiredCapacity" }
},
[…]
},
Cluster Setup with AWS CloudFormation
"ContainerInstances": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"commands" : {
"01_add_instance_to_cluster" : {
"command" : { "Fn::Join": [ "", [ "#!/bin/bash\n", "echo ECS_CLUSTER=", { "Ref": "ECSCluster" }, " >> /etc/ecs/ecs.config" ] ] }
}
},
[…]
}
}
}
Cluster Setup with AWS OpsWorks
One ECS Cluster layer per stack
One cluster can only be associated with one stack
Cluster Setup with AWS OpsWorks
Update OpsWorks IAM role to allow ecs:* actions
Add instances to layer (24/7, time-based, load-based)
Manage security updates, user permission and access
Amazon ECR Setup
Amazon ECR Setup
You have read and write access to the repositories you
create in your default registry, i.e.
<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com
Repository names can support namespaces, e.g. team-
a/web-app.
Repositories can be controlled with both IAM user access
policies and repository policies.
Amazon ECR Setup
# Authenticate Docker to your Amazon ECR registry
> aws ecr get-login
docker login -u AWS -p <password> -e none https://<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com
> docker login -u AWS -p <password> -e none https://<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com
# Create a repository called ecr-demo
> aws ecr create-repository --repository-name ecr-demo
# Push an image to your repository
> docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/ecr-demo:v1
Amazon ECR Docker Credential Helper
Available today - http://bit.ly/25yXdC3
Place the docker-credential-ecr-login binary on your PATH
Set the contents of ~/.docker/config.json file to be:
{ "credsStore": "ecr-login" }
Push and pull images from ECR without docker login
Amazon ECS Infrastructure
Management
Amazon EC2 Simple Systems
Manager (SSM)
Amazon EC2 Simple Systems Manager (SSM)
Use Amazon EC2 SSM to execute commands on container
instances, e.g. yum update
• Add AmazonEC2RoleForSSM
to instances IAM role to
process Run Commands
• Install SSM Agent
• Create SSM document –
similar to CloudInit userdata
• Lock down AWS-* documents
Monitoring & Logging
Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and
recorded for a period of two weeks
Available metrics: CPUReservation, MemoryReservation,
CPUUtilization, MemoryUtilization
Available dimensions: ClusterName, ServiceName
Monitoring with Amazon CloudWatch
Monitoring with Amazon CloudWatch
Monitoring with Amazon CloudWatch
Use the Amazon CloudWatch Monitoring Scripts to monitor
additional metrics, e.g. disk space:
# Edit crontab
> crontab -e
# Add command to report disk space utilization to CloudWatch every five minutes
*/5 * * * * <path_to>/mon-put-instance-data.pl --disk-space-util --disk-space-used --disk-space-avail --disk-path=/ --from-cron
Configuring Logging in Task Definition
logConfiguration task definition parameter
Requires version 1.18 or greater of the Docker Remote API
Maps to docker run --log-driver option
Log drivers: json-file, syslog, journald, gelf, fluentd,
awslogs
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "awslogs-wordpress",
"awslogs-region": "us-west-2"
}
}
CloudWatch Logging with awslogs
Amazon CloudWatch Logs
Amazon CloudWatch Logs
Amazon CloudWatch Logs
Amazon CloudWatch Logs
Amazon S3
Amazon Kinesis
AWS Lambda
Amazon Elasticsearch Service
Amazon ECS Store
Stream
Process
Search
Logging Amazon ECS API with AWS CloudTrail
{
"eventVersion": "1.03",
"userIdentity": {…},
"eventTime": "2015-10-12T13:57:33Z",
"eventSource": "ecs.amazonaws.com",
"eventName": "CreateCluster",
"awsRegion": "eu-west-1",
"sourceIPAddress": "54.240.197.227",
"userAgent": "console.amazonaws.com",
"requestParameters": {
"clusterName": "ecs-cli"
},
Logging Amazon ECS API with AWS CloudTrail
"responseElements": {
"cluster": {
"clusterArn": "arn:aws:ecs:eu-west-1:560846014933:cluster/ecs-cli",
"pendingTasksCount": 0,
"registeredContainerInstancesCount": 0,
"status": "ACTIVE",
"runningTasksCount": 0,
"clusterName": "ecs-cli",
"activeServicesCount": 0
}
},
[…]
Monitoring Amazon ECS with Datadog
Monitoring Amazon ECS with Sysdig Cloud
Scaling Amazon ECS
Setup ECS Cluster with AutoScaling
Create LaunchConfiguration
• Pick instance type
depending on resource
requirements, e.g. memory
or CPU
• Use latest Amazon Linux
ECS-optimized AMI, other
distros available
Create AutoScaling group and
set to cluster initial size
Auto Scaling your Amazon ECS Cluster
Create CloudWatch alarm
on a metric, e.g.
MemoryReservation
Configure scaling policies to
increase and decrease the
size of your cluster
Auto Scaling your Amazon ECS services
Auto Scaling your Amazon ECS services
Service Discovery &
Configuration Management
Service Discovery with ECS Services & Route 53
Route 53 private hosted zone
Set search path on hosts with DHCP option sets
Define ECS services with ELB
Create CNAMEs for each ELB
Service Discovery with ECS Services & Route 53
Task
Task TaskTask
ECS
Service
Application
router, e.g. nginx
Internal ELB with
CNAME, e.g. api.example.com
Route 53 private
zone, e.g. example.com
Service Discovery with Weaveworks
DNS interface for cross-host
container communication
Gossip protocol to share
grouped updates
Overlay network between hosts
Service Discovery and Configuration
Management with ConsulThree main components:
• Consul agent - Runs on each node, responsible for checking the health of the services and of the node itself.
• One or more Consul servers - Store and replicate data, leader elected using the Raft consensus algorithm
• Registrator agent - Automatically register/deregisters services based on published ports and metadata from the container environment variables defined in the ECS task definition
Service Discovery and Configuration
Management with Consul
EC
S C
lus
ter
consul-server
ECS Instance
consul-agent
registrator
ECS Instance
Back end 1
Back end 2
consul-agent
registrator
ECS Instance
Front end
EC
S C
lus
ter
Service Discovery and Configuration
Management with etcd
etcd
registrator
ECS Instance
Container 1
Container 2
confd etcd
registrator
ECS Instance
Container 1
Container 2
confd etcd
registrator
ECS Instance
Container 1
Container 2
confd
Security
ECS IAM Policies and Roles
The ECS agent calls the ECS APIs on your behalf, so
container instances require an IAM policy and role that
allows these calls.
The ECS service scheduler calls the EC2 and ELB APIs on
your behalf to register and deregister container instances
with your load balancers.
Use AmazonEC2ContainerServiceforEC2Role and
AmazonEC2ContainerServiceRole managed policies
(respectively)
ECR IAM Policies and Roles
ECR uses resource-based permissions to control access.
By default, only the repository owner has access to a
repository.
You can apply a policy document that allows others to
access your repository.
Use managed policies for IAM users or roles that allow
differing levels of control:
AmazonEC2ContainerRegistryFullAccess,
AmazonEC2ContainerRegistryPowerUser or
AmazonEC2ContainerRegistryReadOnly
Image Vulnerability Scanning with Twistlock
Secrets Management
Task
ECS Cluster
Container instance
PaaS on ECS
AWS Elastic Beanstalk
Uses Amazon ECS to coordinate deployments to
multicontainer Docker environments
Takes care of tasks including cluster creation, task
definition and execution
AWS Elastic Beanstalk
Elastic Beanstalk uses a Dockerrun.aws.json file that
describes how to deploy containers.
The Dockerrun.aws.json file includes three sections:
• AWSEBDockerrunVersion: Set to "2" for multicontainer
Docker environments.
• containerDefinitions: An array of container definitions.
• volumes: Creates mount points in the container instance
that a container can use.
Convox
Convox
# Initialize your app and create default manifest
> convox init
# Locally build and run your app as declared in the manifest
> convox start
# Create app
> convox apps create my_app
# Deploy app, output ELB DNS name
> convox deploy
[...]
web: http://my_app-1234567890.us-east-1.elb.amazonaws.com
Remind Empire
Control layer on top of Amazon ECS that provides a
Heroku like workflow
Any tagged Docker image can be deployed to Empire as
an app
• When you deploy a Docker image to Empire, it will
extract a Procfile from the WORKDIR
• Each process type in the Procfile maps directly to an
ECS Service
Remind Empire
Routing layer backed by internal ELBs
• An application that specifies a web process will get an
internal ELB attached to its ECS Service
• When a new internal ELB is created, an associated
CNAME record is created in Route53 under the internal
TLD, enabling service discovery via DNS
Thank You!