HU-Stack: Eine cloud-basierte Plattform als flexible ... · Image Editing Mail Service Mailing...

transcript

HU-Stack: Eine cloud-basierte Plattform als flexible Service-Infrastruktur

Malte Dreyer

Profil / Struktur

Computer- und Medienservice (CMS)

Verwaltung (zentral)

• Ressourcen-Mgmt.• Campus-Mgmt.• Verwaltungs PCs

Hardware / Software• Endgeräte (~3.000)• SW-Lizenz-Mgmt.• Benutzerberatung

Digitale Medien• Medien-Mgmt.• Lernumgebungen• Audio-/Video-Technik

Basis-Infrastruktur• Server• Storage• Netze

„Nachwuchsgruppen“„Nachwuchsgruppen“

Profil / Struktur

Computer- und Medienservice (CMS)

Verwaltung (zentral)

• Ressourcen-Mgmt.• Campus-Mgmt.• Verwaltungs PCs

Hardware / Software• Endgeräte (~3.000)• SW-Lizenz-Mgmt.• Benutzerberatung

Digitale Medien• Medien-Mgmt.• Lernumgebungen• Audio-/Video-Technik

Basis-Infrastruktur• Server• Storage• Netze

„Nachwuchsgruppen“„Nachwuchsgruppen“„Nachwuchsgruppen“

Projekte und Plattformen(auch Forschungsdatenmanagement)

AG ePublishing(gemeinsam mit UB)

Basisinfrastruktur

10.09.2013

Computer- und Medienservice4

Zusammenarbeit mit den Einrichtungen

Servicetiefe des CMS

Plattformen

Dienste

Basis-Infrastruktur

Arbeitsplätze

Computer-und

Medien-Service

Computer-und

Medien-Service

Inst. 2

Computer-und

Medien-Service

Inst. 3

Computer-und

Medien-Service

ZentraleVerwaltung

Inst. 1

Breites Diensteportfolio

Identity Management Service

Image Editing

Mail Service

Mailing Lists

Map Service

Moodle

Printing Service

Public Computers

Publication Repository

Resource Management SW

Scanning Service

Shibboleth

Software Service

Statistics SW Service

Survey

Terminalserver Service Service

Professional Training

Trouble Ticket Systems

User Certificates

End User Support

Virtual Private Networks

Video Conference Service

Video Service

Web Content Management

Web Files

Web Hosting

Website Management

Wiki and Blog Farms

Windows Domains

ePublishing

Accounts

Anti Virus Software

Audio / Video Equipment

Backup-Service

Calendar Service

Campus Management SW

Chat Service

Compute Clusters

Database Service

Managed Desktops

File Exchange

File Service

Graphics

Hardware Service

Breites Diensteportfolio

…aber noch nicht breit genug -> Forschungsdaten

Identity Management Service

Image Editing

Mail Service

Mailing Lists

Map Service

Moodle

Printing Service

Public Computers

Publication Repository

Resource Management SW

Scanning Service

Shibboleth

Software Service

Statistics SW Service

Survey

Terminalserver Service Service

Professional Training

Trouble Ticket Systems

User Certificates

End User Support

Virtual Private Networks

Video Conference Service

Video Service

Web Content Management

Web Files

Web Hosting

Website Management

Wiki and Blog Farms

Windows Domains

ePublishing

Accounts

Anti Virus Software

Audio / Video Equipment

Backup-Service

Calendar Service

Campus Management SW

Chat Service

Compute Clusters

Database Service

Managed Desktops

File Exchange

File Service

Graphics

Hardware Service

Biographical Sketch(es), has been revised to rename the “Publications”

section to “Products” and amend terminology and instructions accordingly.

(c) Products

A list of: (i) up to five products most closely related to the proposed

project; and (ii) up to five other significant products, whether or not

related to the proposed project. Acceptable products must be citable

and accessible including but not limited to publications, data sets,

software, patents, and copyrights. Unacceptable products are

unpublished documents not yet submitted for publication, invited

lectures, and additional lists of products. Only the list of 10 will be used

in the review of the proposal.

Each product must include full citation information including (where

applicable and practicable) names of all authors, date of publication or

release, title, title of enclosing work such as journal or book, volume,

issue, pages, website and Uniform Resource Locator (URL) or other

Persistent Identifier.

National Science Foundation (NSF)

Biographical Sketch(es)

NSF 13-1 January 2013

http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/gpg_2.jsp#IIC2d

1. Recommended open access to scholarly papers of

publicly funded research

2. Recommended open access to all digital outputs of

3. Mandated open access to scholarly papers of publicly

funded research

4. Mandated open access to all digital outputs of

5. Enforced, mandated open access to scholarly papers

of publicly funded research

6. Enforced, mandated open access to all digital

outputs of publicly funded research

By Muriel (originally posted to Flickr as Muon Barrel) [CC-BY-SA-2.0 (http://creativecommons.org/licenses/by-

sa/2.0)], via Wikimedia Commons

By Hannes Grobe (Hgrobe 06:16, 26 April 2006 (UTC)), Alfred Wegener Institute for Polar and Marine Research

(Own work) [CC-BY-SA-2.5 (http://creativecommons.org/licenses/by-sa/2.5)], via Wikimedia Commons

Small Data

„Long Tail of Research Data“

Source: Ferguson et al. (2014): Big data from small data: data-sharing in the

'long tail' of neuroscience. DOI: 10.1038/nn.3838

Science 11 February 2011: Vol. 331 no. 6018 pp. 692-693 DOI: 10.1126/science.331.6018.692

The 2011 survey by Science, found that 48.3% of respondents were working with datasets that were less than 1GB in size and over half of those polled store their data only in their laboratories.

Simukovic, Elena; Kindling, Maxi; Schirmbacher, Peter (2013): Umfrage zum Umgang mit digitalen Forschungsdaten an der

Humboldt-Universität zu Berlin. Umfragebericht, Version 1.0. URN: urn:nbn:de:kobv:11-100213001n=457

https://www.cms.hu-berlin.de/de/ueberblick/projekte/dataman/home

Anpassungen und Abbildung

von Szenarien

z. B. Medien-Repositorien-Farm

Gemeinsame Projekte und

Software-Entwicklung

z. B. Laudatio Corpus Repositorium

Projekte in enger Zusammenarbeit mit den Instituten

(gemeinsame Drittmitteleinwerbung)

80 Individuelle Projekte

80 Installationen

80 Datenbanken

80 Konfigurationen

Individuelle Software

Fedora Repository

CakePHP

Elastic Search

PID-Server (Handle)

jQuery / Bootstrap

Apache Xerces

+RedMine for Tickets

Aus Sicht des Datenzentrums

... Nutzer erwarten dauerhaften Betrieb …

80 Individual Projets

80 Installations

80 Databases

80 Configurations

Individual Software

Fedora Repository

CakePHP

Elastic Search

PID-Server (Handle)

jQuery / Bootstrap

Apache Xerces

+RedMine for Tickets

Aus Sicht des Datenzentrums

... Pflegeaufwand …

Änderungen und Erweiterungen

(Maßkonfektion)

Überwiegende Eigenentwicklung(Maßgeschneidert)

Selbstadministration / Selbstbedienung

By Paul Hudson from United Kingdom (11" MacBook Air) [CC BY 2.0

(http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

By HLundgaard (Own work) [CC BY-SA 3.0

(http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

By Sae1962 (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

Flexibilisierung der Infrastruktur

XaaS „Anything as a Service“

Selbstbedienungsfunktionen (cloud)

IaaSInfrastructure

PaaSPlatforms

SaaSSoftware /

Applications

HardwareServer / Storage /

Network

IaaSInfrastructure

PaaSPlatforms

Cloud computing is a model for enabling ubiquitous,

convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released withminimal management effort or service provider interaction.

The NIST Definition of Cloud Computing

(US-National Institute of Standards and Technology SP 800-145)

Automatisierung /

Mehr Softwareentwicklung und –Technik

By Mixabest [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-

sa/3.0), via Wikimedia Commons

Das Virtuelle Rechenzentrum

my_port01:

type: OS::Neutron::Port

properties:

admin_state_up: true

network_id: { get_resource: my_first_network }

my_server07:

type: OS::Nova::Server

properties:

key_name: { get_param: keypair_name }

flavor: { get_param: machine_flavor }

image: { get_param: image_id }

networks: [{ "port": { get_resource: my_port07 }, …}]

user_data: |

#!/bin/bash -v

cinder_volume07:

type: OS::Cinder::Volume

properties:

size: { get_param: volume_size }

volume_attachment07:

type: OS::Cinder::VolumeAttachment

properties:

volume_id: { get_resource: cinder_volume07 }

instance_uuid: { get_resource: my_server07 }

mountpoint: /dev/vdb

ANSIBLE Playbook

# update packet cache

- hosts: all

tasks:

- apt:

- name: apply common configuration to all nodes

hosts: all

roles:

- common

- name: deploy and configure memcached server

hosts: memcached

roles:

- memcached

- name: deploy MySQL and configure the databases

hosts: db

roles:

- name: deploy seafile and apply configuration

hosts: nodes

roles:

- nodes

- name: deploy loadbalancer

hosts: loadbalancer

roles:

- loadbalancer

Computer- und Medienservice

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

OpenStack Komponenten

• Preise [TEUR]

2016 2017 2018

WDM (Licht) 200 200

Linecards 10 Gbps x20

MLX (Router) Chassis 8x

175 175

Linecards(26) 10Gbps x8

286 286

Summe 661 661 200

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Technische Komponenten der Cloud / Organisatorische Einbettung

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Horizon

Dashboard

Compute

Neutron

Network

Object Storage

Glance

Image Mgmt

Orchestration

Ceilometer

Metering

Keystone

Identity Mgmt

Cinder

Block Storage

Database Mgmt

Sahara

Hadoop

Mein Virtuelles Rechenzentrum

my_port01:

type: OS::Neutron::Port

properties:

admin_state_up: true

network_id: { get_resource: my_first_network }

my_server07:

type: OS::Nova::Server

properties:

key_name: { get_param: keypair_name }

flavor: { get_param: machine_flavor }

image: { get_param: image_id }

networks: [{ "port": { get_resource: my_port07 }, …}]

user_data: |

#!/bin/bash -v

cinder_volume07:

type: OS::Cinder::Volume

properties:

size: { get_param: volume_size }

volume_attachment07:

type: OS::Cinder::VolumeAttachment

properties:

volume_id: { get_resource: cinder_volume07 }

instance_uuid: { get_resource: my_server07 }

mountpoint: /dev/vdb

ANSIBLE Playbook

# update packet cache

- hosts: all

tasks:

- apt:

- name: apply common configuration to all nodes

hosts: all

roles:

- common

- name: deploy and configure memcached server

hosts: memcached

roles:

- memcached

- name: deploy MySQL and configure the databases

hosts: db

roles:

- name: deploy seafile and apply configuration

hosts: nodes

roles:

- nodes

- name: deploy loadbalancer

hosts: loadbalancer

roles:

- loadbalancer

..…Danke…..

malte.dreyer@cms.hu-berlin.de

Fragen?

malte.dreyer@cms.hu-berlin.de

Computer- und Medienservice

Backup

HU-Stack: Eine cloud-basierte Plattform als flexible ... · Image Editing Mail Service Mailing...

Documents