+ All Categories
Home > Documents > DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm...

DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm...

Date post: 26-May-2020
Category:
Upload: others
View: 4 times
Download: 0 times
Share this document with a friend
41
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Steffen Grunwald, Solutions Architect, AWS 18. May 2017 DevOps at Amazon: A Look at Our Tools and Processes
Transcript
Page 1: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Steffen Grunwald, Solutions Architect, AWS

18. May 2017

DevOps at Amazon: A Look at Our Tools and Processes

Page 2: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

15 years

Page 3: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

2/3 IT budget

Page 4: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

77% CEOs

Page 5: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Responding requires a new model

Focus on differentiating your company

Innovate at start-up like speed

Reduce risk

Page 6: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

AWS Pace of Innovation

61

516

1017

159

2010 2012 2014 2016

Page 7: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

3,235AWS Direct

Connect

AWS Elastic Beanstalk

Schema Conversion Tool

AWS Shield EFS

WorkSpaces

Amazon Lumberyard

AmazonPinpoint

AWS IoT

AWS Managed ServicesAmazon Route 53

AWS Import/Export

AWS OpsWorks forChef Automate

Redshift

Dynamo DB

Amazon Polly

AWS Snowball

AWS Organizations

Device Farm

Amazon Config

Amazon RDS for Aurora

WorkDocs

AWS Snowball Edge

CodeCommit

AWS CodePipeline

AWS Service Catalog

CloudWatch Logs

Amazon Lex

AWS Greengrass

Amazon EC2Systems Manager

AWS WAF

Amazon Appstream 2.0

Amazon Athena

AWS Glue

Amazon Lightsail

Amazon Rekognition

AWS Step Functions

AWS DiscoveryServices

AWS CertificateManager

Amazon ElastiCache

Mobile Analytics

AWS Mobile Hub

AWS Storage GatewayAWS OpsWorks

AWS BatchAmazon Inspector

EC2Container Service

Amazon Cognito

AWS CodeDeploy

AWS Personal Health Dashboard

AWS Snowmobile

Lambda

* As of 1 May 2017

AWS Codebuild

AWS X-Ray

Amazon QuickSight

Amazon Kinesis Firehose

Amazon Workmail

Amazon Inspector

Machine Learning

Page 8: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

But innovation is hard…

…how can companies build an effectiveinnovation system and an environment

that will foster and support human creativity and drive technological

progress?

Page 9: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

f(innovation) = (culture * structure) tooling

(Innovation is the product of culture and structure with the power of tooling)

Page 10: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

f(innovation) = (culture * structure) tooling

Page 11: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Innovation requires culture that fosters invention.

Culture is the principal componentin velocity of innovation.

Page 12: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Amazon leadership principles

https://www.amazon.jobs/principles

Page 13: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

A company of builders…builders come to build.

Page 14: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

“Invention requires two things: the ability to try a lot of experiments, and not having to live with

the collateral damage of failed experiments”

Andy Jassy, CEO, Amazon Web Services

Page 15: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

f(innovation) = (culture * structure) tooling

Page 16: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Good intentions are never enough, you must build structure and create

innovation fostering habits.

Page 17: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

We measure performance of our processes, improve it and

remove bottlenecks.

Page 18: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

A world of conflicting priorities

DevelopersPaid tochange

SecurityPaid to prevent

risk

OperationsPaid to ensure

stability

Page 19: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

…and of bottlenecks

Development Testing Security Operations

Page 20: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Avoid future firefighting by including others early and by investing time to

paying back technical debt.

Page 21: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

2001

Development transformation at Amazon: 2001-2009

20092006

Page 22: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

“If the development team is frequently called in the middle of the night, automation is the likely outcome.If operations is frequently called, the usual reaction is to grow the operations team.”

James Hamilton, Distinguished Engineer, Amazon Web Services

Page 23: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

While all teams are autonomous, they are defined and driven by the cultural DNA (Leadership Principles) at every step.

Page 24: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Strive for continuous deployment.Use metrics and tooling to gain trust.

Continuous integration

Continuous delivery

Continuous deployment

Page 25: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Continuous Deployment Benefits

Improve developer productivityAutomate the software release process

Find and address bugs quickly

Deliver updates faster

Page 26: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

f(innovation) = (culture * structure) tooling

Page 27: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Tooling should be decentralised, encouraging self service.

Page 28: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

It should promote best practices without being restrictive.

Page 29: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

It should be technology agnostic.

Page 30: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

It should be the path of least resistance.

Page 31: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Continuous Delivery

From check-in to production

CI/CD + Release Automation

>90% of Amazon teams

Pipelines

Page 32: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

12 years young

Rolling Deployments (zero downtime)

Health Checking

Versioned Artifacts & Rollbacks

Page 33: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

= 50 million deployments a year

Thousands of teams +Microservices architectures +Multiple environments +Continuous delivery?

Page 34: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

From epic feature releasesto many tiny releases.

Page 35: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

AWS services help youdo the same.

Page 36: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Deployment, Administration & Monitoring

MonitorDeployProvisionTestBuildCode

AWS Elastic Beanstalk/ -Container Service

Serverless/ AWS LambdaAmazon

CloudWatchAWS CloudFormation

AWS CodeDeploy

AWS CodeCommit

AWS CodePipeline

AWS CodeBuild

Page 37: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

AWS CodeStar: Everything you need to Develop, Build, and Deploy

applications on AWS.

Page 38: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

AWS CodeStar Project Templates

Page 39: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Demo Time!

Page 40: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

f(innovation) = (culture * structure) tooling

Page 41: DevOps at Amazon: A Look at Our Tools and Processes · AWS Snowball AWS Organizations Device Farm Amazon Config Amazon RDS for Aurora WorkDocs AWS Snowball Edge CodeCommit AWS CodePipeline

Turn your innovative idea into reality. Start by spinning up all you

need with AWS CodeStar.


Recommended