DLF Fall Forum 2012, Tales from the Cloud

Our Bits Sit on AWSTexas Digital Library and Amazon

Ryan SteansSenior Program Coordinator, Texas Digital Library

[email protected]

Agenda

• Who we are• Fall 2010 – where we were• Why we looked at Amazon• Spring 2011• Today and Looking Forward

TDL DLF Presentation

What is Texas Digital Library?


• 14 Member Consortium of Libraries at academic/ research institutions across Texas

• TDL provides centralized software support and development• Cost share• Provide scholarly communication

options to schools all at the same level

Texas Digital Library Services


• DSpace internet repositories (DuraSpace)• Vireo ETD Management System (TDL)• Open Journal System (PKP)• Open Conference System (PKP)• WordPress for research group websites and

blogs• MediaWiki for research group data sharing and

management• Training, consultation, conference, community

TDL Activities – in the works


• Digital Preservation Network• Fedora/ Hydra• Data management planning• Shared research repositories/ communities• Vireo users community (thesis and dissertation

community)

Fall 2010


• TDL located on UT Austin Campus• TDL hardware at UT Data Center• Data Center move scheduled for December

2010 for all of UT campus• TDL hardware was very close to expiration

of service contracts, etc…• TDL began looking at AWS for a backup

option during the move


Texas Digital Library Inventory - AustinMake Model QuantityUsage

Servers Sun SunFire 490 2Database / Application Server / DNS / LDAP / Handle / Shib

Sun Sunfire T5220 2Dspace / WWW / Blogs / Wikis / Faculty Pages / Journals / Mail / Services / Support / SVN

Sun Sunfire 245 2 Dspace Labs / DevelopmentSun Sunfire T5120 1 Preservation (iRods/SAM-QFS)

Sun Sunfire X4150 2Old content management servers (positioned to help w/ web applications )

NetworkCisco Cataylst 3750 2 Network connectivityCitrix Netscaler 2 Load BalancersBrocadeSilkworm 300E 2 SAN ConnectivityAvocentCycladed ACS 2 Serial Console Connectivity

Storage NetApp Filer 3020 2Dspace Assets / Web Application Data / Home Directories / Backup

Sun StorageTek 614 1Preservation (iRods / SAM-QFS) / Virtual Machines / Databases

Sun SL500 1 Preservation (iRods / SAM-QFS)

Texas Digital Library Inventory - College StationServers Sun T 5220 2 Dspace

Sun T 5120 1 Preservation

StorageNetApp Filer 3020 1 Dspace Assets / Backup

2010 ‐ Our existing hardware


• Very locked down with space• Had to over‐buy in order to guess need• Limited staff was worrying about physical

hardware• All eggs in one basket• Tied to university security protocols that were

difficult with multiple members• IT infrastructure had led to bad user experience –

shared hardware, machines, etc…

Looking at Amazon Web Services


• ELASTIC• Could increase size of a machine• Could increase number of machines• Removed guess work and bad estimates

from the equation• Elastic cost

• Service Contracts were highly reasonable• Road tested. AWS launched in 2006.

Spring 2011


• We moved to AWS for all services

Why?


• Allowed us to separate service instances, bring each up on a VM

• Created far greater user customization/ access/ control

• Rapid deployment of services from master image

• Accounting

How?


• Set up a duplicate system in AWS• Used load balancer as the fulcrum• Caused a minimum of downtime

Today


• Running 100 separate AWS instances as of 11/19/2012

• Deploying services easily• Literally nothing is on UT’s hardware nor at

any TDL member school• Depend on standard security and AWS• Cost


It’s actually working just fine


• A bit of “beg forgiveness instead of asking permission”

• Costs are down• Members don’t notice• Our staff is handling it well (2 longtime

university IT people as sys admins)

An overly simple idea of service provisioning


But…


• Network cost could become burdensome• Cost savings per unit no longer as obvious• UT and our own team more comfortable

with VMs and could move to purchased hardware

• AWS billing not easy to manage month‐to‐month

And…


• As we look at the Digital Preservation Network ‐ we will rebuild on campus• Security concerns• Goal of group is to do this in academia• It may also affect how we think of

implementing Fedora/ Hydra

Week of October 22nd…


• TDL learned of outage at 1:15 PM Central• Services were visible, but not accessible• Contacted members• Watched logs, news reports, etc…• 7:30 PM we were back up and running

with no data loss• Received 7 communications through

helpdesk

It’s bad, but…


• AWS was in communication with users• We feel confident in them, and they got it

back up• Checking our systems was fairly easy• Took some pressure off our staff• Outages are not limited to AWS

Looking forward


• We will always rethink our strategy on AWS

• AWS is providing us with a sales person and a consultant before end of year

• Cost has to remain reasonable for uploading, downloading

• Will be moving preservation efforts to campus

Ryan Steans, Program CoordinatorKristi Park, Program Coordinator

[email protected]

Andrew Woodshttp://duracloud.org

04 Nov 2012

Tales from the Cloud: Experiences and Challenges

http://duracloud.org/

2

Why might I even be interested in “the cloud”

?

3

What are some risks

?

4

How is it different from local infrastructure

?

5

What about this “DuraCloud”

?

Date post:	18-Dec-2014
Category:	Technology
Upload:	duraspace
View:	2,070 times
Download:	1 times

DLF Fall Forum 2012, Tales from the Cloud

Technology