Date post: | 16-Apr-2017 |
Category: |
Technology |
Upload: | amazon-web-services |
View: | 2,331 times |
Download: | 1 times |
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chris BarclayPrincipal Product Manager
Running Batch Jobs on Amazon EC2 Container Service
Agenda
Containers
EC2 Container Service
Common patterns
Demo
Q&A
Containers
What are Containers?
OS virtualization
Process isolation
Images
Automation Server
Guest OS
Bins/Libs Bins/Libs
App2App1
Container Advantages
PortableServer
Guest OS
Bins/Libs Bins/Libs
App2App1
Container Advantages
FlexibleServer
Guest OS
Bins/Libs Bins/Libs
App2App1
Container Advantages
FastServer
Guest OS
Bins/Libs Bins/Libs
App2App1
Container Advantages
EfficientServer
Guest OS
Bins/Libs Bins/Libs
App2App1
Server
Guest OS
Bins/Libs Bins/Libs
App2App1
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
EC2 Container Service Benefits
Easily Manage Clusters for Any Scale
Nothing to run
Complete state
Control and monitoring
Scale
Flexible Container Placement
Applications
Batch jobs
Multiple schedulers
Designed for Use with Other AWS Services
Elastic Load Balancing
Amazon Elastic Block Store
Amazon Virtual Private Cloud
AWS Identity and Access Management
AWS CloudTrail
Extensible
Comprehensive APIs
Open source agent
Custom schedulers
EC2 Container Service Terminology
Amazon EC2 instances
Docker daemon
Amazon ECS agent
Key Components: Container Instances
Regional
Resource pool
Grouping of Container Instances
Start empty, dynamically scalable
Key Components: Clusters
Key Components: Task Definitions
Volume Definitions
Container Definitions
Key Components: Task Definitions
Shared Data Volume
PHP App Time of day App
Key Components: Task Definitions { "environment": [], "name": "simple-demo", "image": "my-demo", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my-vol" } ], "entryPoint": [ "/usr/sbin/apache2", "-D", "FOREGROUND" ], "essential": true },
{ "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom": [ { "sourceContainer": "simple-demo" } ], "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"while true; do /bin/date > /var/www/my-vol/date; sleep 1; done\"" ], "essential": false }
{ "environment": [], "name": "simple-demo", "image": "my-demo", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my-vol" } ], "entryPoint": [ "/usr/sbin/apache2", "-D", "FOREGROUND" ], "essential": true },
Key Components: Task Definitions[ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] }]
Essential to our Task
Create and mount volumes
Expose port 80 in containerto port 80 on host
10 CPU Units (1024 is full CPU),500 Megabytes of Memory
{ "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom": [ { "sourceContainer": "simple-demo" } ], "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"while true; do /bin/date > /var/www/my-vol/date; sleep 1; done\"" ], "essential": false }
Key Components: Task Definitions[ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, ]
From Docker Hub
Mount volume from other container
Command to exec
Key Components: Tasks
Container Instance
Schedule
Shared Data Volume
PHP App Time of day App
Shared Data Volume
PHP App Time of day App
Unit of work
Grouping of related Containers
Run on Container Instances
Key Components: Tasks
Key Components: Run a Task
Good for short-lived containers, e.g.
batch jobs
Key Components: Create a Service
Good for long-running applications
and services
Key Components: Create Service
Load Balance traffic across containersAutomatically recover unhealthy containersDiscover services
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Elastic Load Balancing
Key Components: Update Service
Scale upScale down
Elastic Load Balancing
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Key Components: Update Service
Deploy new versionDrain connections
Elastic Load Balancing
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Key Components: Update Service
Deploy new versionDrain connections
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Elastic Load Balancing
Key Components: Update Service
Deploy new versionDrain connections
Shared Data Volume
PHP App
Time of day App
Shared Data Volume
PHP App
Time of day App
Elastic Load Balancing
Shared Data Volume
PHP App
Time of day App
Batch jobs
Containers and Batch Jobs
Easy to update
Any app, any language
Includes all dependencies
Image is the version
Test & deploy same artifact
When to Use ECS and Lambda
ECSLong-running jobs
Manage your event triggering
Any language, any dependency
Resources are your own - use
Spot, RIs
LambdaShort-lived jobs
Triggered on specific events
Supports specific environments
No infrastructure to manage
Demo: Batch Jobs with ECS
.zip
s3://bucket/scene.zip
ecs:RunTask
ECS Cluster
.png
Demo
Q&A
AWS re:Invent 2015 – October 6-9AWS re:Invent is the largest annual gathering of the global cloud community. Whether you are an existing customer or new to the cloud, AWS re:Invent will provide you with the knowledge and skills to refine your cloud strategy, improve developer productivity, increase application performance and security, and reduce infrastructure costs.
Though AWS re:Invent tickets are sold out, you can still register to view the Live Stream Broadcasts of the keynote addresses and select technical sessions on October 7 and October 8. Register now.
Details:Wednesday, October 79:00am - 10:30am PT: Andrew Jassy, Sr. Vice President, AWS11:00am - 5:15pm PT: 5 of the most popular breakout sessions (to be announced)
Thursday, October 89:00am - 10:30am PT: Dr. Werner Vogels, CTO, Amazon11:00am - 6:15pm PT: 6 of the most popular breakout sessions (to be announced)
Register now for the Live Stream Broadcast by submitting your email where prompted on the AWS re:Invent home page.
Stay Connected: Follow event activities on Twitter @awsreinvent (#reinvent), or like us on Facebook.
Thank you!