+ All Categories
Home > Technology > (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

(APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Date post: 27-Jun-2015
Category:
Upload: amazon-web-services
View: 640 times
Download: 2 times
Share this document with a friend
Description:
"AWS OpsWorks helps you deploy and operate applications of all shapes and sizes. With AWS OpsWorks, you can model your application stack with layers that define the building blocks of your application: load balancers, application servers, databases, etc. But did you know that you can also extend AWS OpsWorks layers or build your own custom layers? Whether you need to perform a specific task or install a new software package, AWS OpsWorks gives you the tools to install and configure your instances consistently and help them evolve in an automated and predictable fashion. In this session, we dive into the development process including how to use attributes, recipes, and lifecycle events; show how to develop your environment locally; and provide troubleshooting steps that reduce your development time."
Popular Tags:
62
November 12, 2014 | Las Vegas, NV Thomas Lobinger and Jonathan Weiss, Amazon Web Services
Transcript
Page 1: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

November 12, 2014 | Las Vegas, NV

Thomas Lobinger and Jonathan Weiss, Amazon Web Services

Page 2: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 3: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 4: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 5: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

OpsWorks

Backend

OpsWorks

Agent

Amazon EC2, Amazon EBS, EIP,

Amazon VPC, Elastic Load Balancing….

Auto-Scaling, Auto-Healing….

On-instance execution via

Chef client/zero

Command

JSON

Command

Log+Status

Page 6: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Chef Client

&

Chef Server

Page 7: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 8: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 9: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 10: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 11: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 12: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 13: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 14: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 15: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

AWS OpsWorks

recipes

Yourcustomrecipes

Page 16: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 17: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 18: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

search(:node, “name:web1”)

search(:node, “name:web*”)

Page 19: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

appserver = search(:node, "role:php-app").first

Chef::Log.info(”Private IP: #{appserver[:private_ip]}")

Page 20: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

appserver = search(:node, "role:php-app").first

Chef::Log.info(”Private IP: #{appserver[:private_ip]}")

Page 21: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

{"opsworks": {

"data_bags": {"bag_name1": {

"item_name1: {"key1" : “value1”,"key2" : “value2”,...

}},"bag_name2": {

"item_name1": {"key1" : “value1”,"key2" : “value2”,...

}},...

}}

}

Page 22: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

{ "opsworks": {

"data_bags": {"myapp": {"mysql": {

"username": "default-user","password": "default-pass"

}}

}}

}

mything = data_bag_item("myapp", "mysql")Chef::Log.info("username: #{mything['username']}")

Page 23: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 24: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

user = ENV[“payment_provider_user“]password = ENV[“payment_provider_password“]

PaymentGateway.new(user, password)

Page 25: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 26: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 27: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 28: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 29: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 30: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 31: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Change

cookbook

Push

change

Stop an

instance

Update

cookbooks

Start an

instance

Trigger

event or

runlist

Check logs

Page 32: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 33: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

list_commands

get_json

run_command

show_log

Page 34: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

list

# list event history (up to 10 events)

# opsworks-agent-cli list

2014-11-06T10:26:58 setup

2014-11-06T10:55:13 configure

2014-11-06T14:59:05 execute_recipes

2014-11-07T18:25:53 deploy

2014-11-07T18:34:11 configure

2014-11-08T10:05:23 configure

2014-11-08T14:43:43 configure

Page 35: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

get

# show JSON events (latest or by type/timestamp)

# opsworks-agent-cli get

{

"ssh_users": {

"2066": {

"name": "thomas",

"public_key": "ssh-rsa AAAAB3NzaC1...",

"sudoer": true

...

Page 36: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

# list current agent folder and cookbooks

# ls /opt/aws/opsworks/current

cookbooks # OpsWorks cookbooks

berkshelf-cookbooks # Cookbooks from Berkshelf

site-cookbooks # Your cookbooks

merged-cookbooks # Resulting cookbooks for run

...

Page 37: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

run

# run last setup event (changed cookbooks, same JSON)

# opsworks-agent-cli run setup

# or select specific date

# opsworks-agent-cli run 2014-11-08T14:43:43

[2014-11-12 13:06:41] INFO [opsworks-agent(17817)]: About to re-run

'setup' from 2014-11-08T14:43:43

Waiting for process 17817

...

Finished Chef run with exitcode 0

Page 38: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

# change recipe

# cd /opt/aws/opsworks/current/site-cookbooks

# vim mycookbooks/recipes/default.rb

# confirm success

# opsworks-agent-cli run setup

# commit back to your git repo

# git commit -am “can develop on instance“

# git push

Page 39: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 40: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 41: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Destroy Create Converge Setup Verify Destroy

Page 42: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

.kitchen.yml

driver:

name: opsworks

stack_id: vvvvvvvv-wwww-xxxx-yyyy-zzzzzzzzzzzz

provisioner:

name: shell

busser:

ruby_bindir: /opt/aws/opsworks/local/bin

Page 43: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

.kitchen.yml

platforms:

- name: Amazon Linux 2014.09

- name: Ubuntu 12.04 LTS

- name: Ubuntu 14.04 LTS

suites:

- name: default

run_list:

- recipe[demo::default]

Page 44: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

# destroy, create, setup, converge, verify, destroy

kitchen test

kitchen verify [NAME|REGEXP|all] # run all 'tests'

kitchen verify ubuntu # only ubuntus

kitchen verify client # only client tests

kitchen verify -c NUMBER # concurrency

# Talk by Fletcher Nichol http://youtu.be/YzlCHAbJ7KM

Page 45: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

‣ integration

.kitchen.yml

metadata.rb

Page 46: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

▾ integration

‣ default <-- test suite name

.kitchen.yml

metadata.rb

Page 47: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

▾ integration

▾ default <-- test suite name

‣ bats <-- test framework

.kitchen.yml

metadata.rb

Page 48: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

‣ attributes

‣ recipes

‣ templates

▾ test

▾ integration

▾ default <-- test suite name

▾ bats <-- test framework

htop_installed.bats <-- the actual test

.kitchen.yml

metadata.rb

Page 49: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

Test file htop_installed.bats

#!/usr/bin/env bats

@test "htop binary is found in PATH" {

run which htop

[ "$status" -eq 0 ]

}

Page 50: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 51: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 52: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 53: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

5

commit

Stack

on

EC2

Page 54: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 55: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 56: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

$./opsrun.rb 92ca56ec-1672-4db0-8c6e-a8566e7db671 'echo $SHELL'ruby: warning: RUBY_HEAP_MIN_SLOTS is obsolete. UseRUBY_GC_HEAP_INIT_SLOTS instead.About to run a script on stack 92ca56ec-1672-4db0-8c6e-a8566e7db671 on 2 online instances......jenkins2:/bin/bashphp-app1:/bin/bash$

Page 57: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 58: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 59: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 60: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 61: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014
Page 62: (APP301) AWS OpsWorks Under the Hood | AWS re:Invent 2014

http://bit.ly/awsevals


Recommended