Post on 10-May-2015
transcript
Automating Your Infrastructure Antons Kranga, OUG Harmony 2012, 1, June 2012
Name: Antons Kranga
Role: Technology Architect, Technology Trainer
Works: Accenture, Member of JUG Latvia
Experience: 10+ in IT, Java
Current assignment: Cloud Computing.
Problem
Copyright © 2012 Accenture All rights reserved. 4
Trends in IT
© behindthecloud.net
Copyright © 2012 Accenture All rights reserved. 5
IT complexity and the timeline
© Mitchell Hill, Opscode,
199
0
199
1
199
2
199
3
199
4
199
5
199
6
199
7
199
8
199
9
200
0
200
1
200
2
200
3
200
4
200
5
200
6
200
7
200
8
200
9
201
0
201
1
201
2
201
3
201
4
201
51980 1981 1982 1983 1984 1985 1986 1987 1988 1989
Virtual Nodes
Physical Hardware
1980 Mainframe
1990 Client/Server
2000 Datacenter
2010+ Cloud
Copyright © 2012 Accenture All rights reserved. 6
Managing IT Complexity
Web Servers
Application
Servers
Database
Copyright © 2012 Accenture All rights reserved. 7
Managing IT Complexity
Web Servers
Application
Servers
Database
+1 server
Copyright © 2012 Accenture All rights reserved. 8
Managing IT Complexity
Web Servers
Application
Servers
Database
20+ changes
Copyright © 2012 Accenture All rights reserved. 9
Take IT Into the Cloud
Copyright © 2012 Accenture All rights reserved. 10
Take IT Into the Cloud
Cloud gives huge
Virtualization
Power
Copyright © 2012 Accenture All rights reserved. 11
Take IT Into the Cloud
Cloud gives huge
Virtualization
Power Power at low cost
Copyright © 2012 Accenture All rights reserved. 12
Take IT Into the Cloud
…and Cloud
Infrastructure
Increases
Copyright © 2012 Accenture All rights reserved. 13
Take IT Into the Cloud
Cost of change
increases
exponentially
Copyright © 2012 Accenture All rights reserved. 14
Take IT Into the Cloud
Changes
increases
exponentially
As well as
infrastructure
complexity
Copyright © 2012 Accenture All rights reserved. 15
Take IT Into the Cloud
How will we
manage our
infrastructure?
Copyright © 2012 Accenture All rights reserved. 16
• Virtualization Efficiency
• Infrastructure and Configuration
Complexity
• Security, Availability etc
• Cloud Provider Lock-in
• Critical Skills Shortage
Challenges of Cloud Adoption
Copyright © 2012 Accenture All rights reserved. 17
Example: Log Collection Becomes Complex
Example of Facebook Scribe
How to survive in the Clouds?
Copyright © 2012 Accenture All rights reserved. 19
Why Automate?
• Scalable
Infrastructure
• Accelerate Changes
• Cloud Provider
Abstraction
• Leverage Skills
Demand
• Manage
Configuration
Copyright © 2012 Accenture All rights reserved. 20
• Tight Integration with Infrastructure
• Enterprise Ready
• Write DSL (not shell commands)
• Promote Reuse
• Active Community
Configuration Management Tools
Chef Overview
Copyright © 2012 Accenture All rights reserved. 22
Infrastructure automation platform
What is Chef?
Copyright © 2012 Accenture All rights reserved. 23
Chef Users write Cookbooks and
Recipes to describe infrastructure in
code
What is Chef?
Copyright © 2012 Accenture All rights reserved. 24
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Copyright © 2012 Accenture All rights reserved. 25
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Written in
Ruby DSL
Copyright © 2012 Accenture All rights reserved. 26
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Resource
Recipe operates
with platform
agnostic resources
Copyright © 2012 Accenture All rights reserved. 27
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Providers Providers
Providers
Resource
Ensures that
configuration
matches to the
platform
Copyright © 2012 Accenture All rights reserved. 28
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Providers Providers
Providers
Resource
Node and
cookbook
configuration
properties
Copyright © 2012 Accenture All rights reserved. 29
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Providers Providers
Providers
Resource
Cookbook
contain other
configuration
information
Copyright © 2012 Accenture All rights reserved. 30
Anatomy of the Cookbook
Cookbook
Recipe Attributes
Configuration
Artifacts
Providers Providers
Providers
Resource
Templates Files Libs
Copyright © 2012 Accenture All rights reserved. 31
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
Chef Server
(private or
hosted)
Copyright © 2012 Accenture All rights reserved. 32
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
Chef Server
(private or
hosted)
May have
WebUI
Copyright © 2012 Accenture All rights reserved. 33
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
Nodes in your
infrastructure,
managed by
chef server
Copyright © 2012 Accenture All rights reserved. 34
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
Stores all nodes
configuration
attributes,
cookbooks etc
Copyright © 2012 Accenture All rights reserved. 35
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
All data
stored in
JSON
Copyright © 2012 Accenture All rights reserved. 36
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
Can be used by
user or
cookbook
Copyright © 2012 Accenture All rights reserved. 37
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
Can be used by
user or
cookbook
Provides powerful
search capabilities
for CoachDB
Copyright © 2012 Accenture All rights reserved. 38
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Stores and
forwards data from
Chef to solr indexer
Copyright © 2012 Accenture All rights reserved. 39
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Stores and
forwards data from
Chef to solr indexer
Updates solr
indexer
Copyright © 2012 Accenture All rights reserved. 40
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Chef Client
All recipes has
been combined in
run list and
executed by chef
client
Copyright © 2012 Accenture All rights reserved. 41
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Chef Client
Chef client uses
for execution “pull”
strategy which is
more flexible in
real life
Copyright © 2012 Accenture All rights reserved. 42
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Chef Client Ohai
Ohai is the tool
to discover
platform
attributes
Copyright © 2012 Accenture All rights reserved. 43
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Chef Client Ohai
User Workstation knife knife
knife
Copyright © 2012 Accenture All rights reserved. 44
Chef Architecture
Chef Server
(Rest API)
Remote Node Remote Node
Remote Node
CouchDB
cookbooks
attributes
databags
…
solr search
RabbitMQ
Chef Client Ohai
User Workstation knife knife
knife
User executes
knife CLI to
operate with Chef
Server
Demo!
Questions?
Reading…
(video) Chef in 5 minutes
http://goo.gl/Of1p5 Cookbook Community
http://goo.gl/yAlPA Cookbooks from 37 Signals
http://goo.gl/uHC7A
Thank You!