Introduction to FutureGrid: Towards a Computing Testbed as a Service

transcript

https://portal.futuregrid.org

Introduction to FutureGrid:Towards a

Computing Testbed as a Service

Gregor von Laszewski

Use my own cluster?

• Pro:– Full access– Can support the research I am interested in

• Con:– Limited scale– Maintenance cost high– Does often do not integrate with multitenancy

research– Limitation of research topics due to availability of

testbed capabilities (often decommissioned hardware)

Use XSEDE-other than FG?

• Pro:– Full managed environment– Software can be added by staff but must undergo production

testing and application need analysis – Scale

• Con:– Focused on application use not CS research– Software stack is “static”– Research limited to application optimization– Targets production software not testing– Resources are mostly dedicated to production– Experiments could impact production

CS Research Testbeds (A personal view)

• Grids:– Wisconsin Grid Testbed: Compile service, limited testing

• HPC:– XSEDE TIS: Focus on XSEDE resources, production

level, limited innovation ability

• EmuLab:– Focus on network research, bare metal

• FutureGrid:– Integrates with bare metal, IaaS, PaaS, limited network

research

What others want on FG

What others want on FGOpenStack

2009-03-22 - 2009-03-282010-07-11 - 2010-07-172011-10-30 - 2011-11-050

openstackcloudstackubuntu euca-lyptusubuntu openstackredhat openstack

Google Trends

Recent Trends

• FG (Project Trends)– All IaaS same interest

volume

– OpenStack – OpenNebula

– Nimbus – Eucalyptus – Eucalyptus (Class)

• Google (User Trends)– OpenStack – CloudStack

– Eucalyptus – Nimbus

Why don’t we support …?

• IU– Model: provisioning by

users and center– IU supports

• HPC (IU&SDSC)• OpenStack• Eucalyptus (IU&SDSC)• Hadoop• various other activities

• UFL– Model: traditional– Cloudstack

• TACC & UC– Model : traditional – HPC– Nimbus

SW Architecture: Integrative View

https://portal.futuregrid.org http://futuregrid.org

Image Generation

• Users who want to create a new FG image specify the following:o OS typeo OS versiono Architectureo Kernelo Software Packages

• Image is generated, then deployed to specified target.

• Deployed image gets continuously scanned, verified, and updated.

• Images are now available for use on the target deployed system.

Provisioning HPC, Grid, and Cloud Services

Management Services

• Image Management• Dynamic Provisioning• Experiment

Management• Monitoring and

Information Services

Select

Features

• < LAPACK• < MongoDB

Generate Imag

• > Meta data• > Raw Image

Store Imag

• > Repository• > Local file systemIf

Management Services

Submit

• < Image• < Job description

Provision

• > Access Image• > Place on Resources

Execute

• > Run Job• > Get Results

Experiment ManagementGoals

• Support rigorous experimentation– Define experiments in detail– Record experimental results

• User-specified measurements (placement and granularity)

– Share experiment information• Experiments can be repeated and verified• Variations on experiments can be performed

• Convenient execution of experiments– FutureGrid has distributed resources and services– Supports different user scenarios

Management Services

Information ServicesTaaS

Experiment

Image Generation

Image Repository

Experiment Repository

Dynamic Provisioning

Nimbus

Eucalyptus

FG Hostlist & Grid5000

TakTuk

Using Rain as Experiment Management Tool

• fg-rain –h hostfile –image img• fg-rain –h hostfile –iaas openstack –image img• fg-rain –h hostfile –paas hadoop …

• Users require not a complex experiment environment, but a high-level interface to it

• We need more than a “workflow” enectment engine

• fg-shell > ….. (part of fg-rain)

http://futuregrid.org

Assemble your own Experiment

• Users have control of the entire stack

• Testbed Production systems adapt based on user and service demand

Adaptive

Services

• Move IaaS nodes• SLA’s

Create

Cluster

• PaaS Provisioning

Seeding

• IaaS Provisioning

Connectivi

• Network Provisioning

Summary

Selected Testbed Capabilities

Pre FG

Application orientedExperiments in production systemsDIY

FutureGrid I

Dynamic Provisioning of OSPreinstalled IaaS(Preinstalled) PaaSVirtual ClustersVirtual Networks (ViNE)DIWU - Do It with us - RAIN

TestBed aaS

Provision Everything (PEG) - Provision IaaS - Provision Paas - Provision NetworkSynergistic Monitor (SMON) - IaaS, OS, Ganglia, NagiosDevice Level Access (DLA)DIAAS - Do It as a service

Introduction to FutureGrid: Towards a Computing Testbed as a Service

Documents