Ocean Observatories InitiativeOcean Observatories Initiative

CEI Demonstrations

Tim Freeman

OOI Cyberinfrastructure Life Cycle ObjectivesMilestone Review, Release 1

San Diego, CAFebruary 23-25, 2010

OOI CI LCO Review, Feb 20102

Overview

• I. Introduction to Nimbus and EC2• II. Contextualization• III. Elasticity

OOI CI LCO Review, Feb 20103

Nimbus and EC2

I. Nimbus and EC2(or: The Basics)

OOI CI LCO Review, Feb 20104

Nimbus and EC2• "Renting" a remote virtual machine

OOI CI LCO Review, Feb 20105

Nimbus and EC2• Demo

– List available images– Launch one– Log in– Do (almost) anything you want– Destroy

OOI CI LCO Review, Feb 20106

Contextualization

II. Contextualization(One-click Clusters)

OOI CI LCO Review, Feb 20107

Contextualization• Problem: start an MPI cluster• The nodes do not stand alone

– machinefile– Security– Often a shared FS

OOI CI LCO Review, Feb 20108

Contextualization• Set it up manually?

– Deal with late binding information

• Each of these tasks needs many commands:– Launch N VMs

– Collect all of their assigned hostnames

– Login to each VM and blindly accept generated SSH host keys

– Login to each VM and retrieve SSH host key

– Login to each VM and configure SSH host based authorization

– Restart all the SSHd processes

– Login to each VM and write machinefile

– Login to one VM and write the shared FS head node policy

– Login to the FS client VMs and write the shared FS client node policy

– Start the FS head node shared FS daemon

– Start the FS client node's shared FS client

– Start the MPI daemons

OOI CI LCO Review, Feb 20109

Contextualization• Structured configuration• Many to many configs• Secure key exchange

OOI CI LCO Review, Feb 201010

Contextualization• Client and VM participation

OOI CI LCO Review, Feb 201011

Contextualization• "Last mile" configurations can be a pain

• (Even worse: each script might be distro specific)

OOI CI LCO Review, Feb 201012

Contextualization• "Last mile" configurations with Chef

OOI CI LCO Review, Feb 201013

Contextualization• Recipes

package "nginx" do action :installend

template "/etc/nginx/nginx.conf" do source "nginx.conf.erb" variables({ :webapp_port => @node[:webapp_port], :webapps => @node[:webapp_ips] })end

service "nginx" do action :restartend

Dynamic Value(Client Provided)

OOI CI LCO Review, Feb 201014

Contextualization• Recipes

package "nginx" do action :installend

template "/etc/nginx/nginx.conf" do source "nginx.conf.erb" variables({ :webapp_port => @node[:webapp_port], :webapps => @node[:webapp_ips] })end

service "nginx" do action :restartend

Dynamic Value(Broker Provided)

OOI CI LCO Review, Feb 201015

Contextualization• Demo

– Launch a load-balancer cluster– Wait for contextualization– Browse to the website– Destroy

OOI CI LCO Review, Feb 201016

Elasticity

III. Elasticity

OOI CI LCO Review, Feb 201017

Elasticity• Nodes are configured automatically, but

how many should there be? When/how long should they run?

• Elasticity– Launch instances as needed– Contract idle instances to save

money/credits/electricity

• Related concepts: reliability/availability– Compensate for errors/crashes– Failover

OOI CI LCO Review, Feb 201018

Elasticity

OOI CI LCO Review, Feb 201019

Elasticity• Demo

– One node already running– Work messages inserted to queue– Sensors report load– Policy evaluated, load unacceptable– More instances are launched– Contraction

OOI CI LCO Review, Feb 201020

Elasticity

Thankyou!