6/12/99 Java Grande T. Haupt 1

The Gateway System

This project is a collaborative effort between • Northeast Parallel Architectures Center (NPAC)

• Ohio Supercomputer Center (OSC)

• Aeronautical Systems Center (ASC) MSRC

6/12/99 Java Grande T. Haupt 2

Goals

• To provide a problem-oriented interface (a Web portal) to more effectively utilize HPC resources from the desktop via the Web browser.

• This “point & click” view hides the underlying complexities and details of the HPC resources and creates a seamless interface between the user’s problem description on his/her desktop system and the heterogeneous computing resources

• These HPC resources include supercomputers, mass storage systems, databases, workstation clusters, collaborative tools, and visualization servers.

6/12/99 Java Grande T. Haupt 3

Design Issues

• Support for a seamless access (security)

• Support for distributed, heterogeneous Back-End services (HPCC, DBMS, Internet, ...) managed independently from Gateway

• Variable pool of resources: support for discovery and dynamical incorporation into the system

• Scalable, extensible, low-maintenance Middle Tier

• Web-based, extensible, customizable, self-adjusting to varying capacities and capabilities of clients (humans, software and hardware) front end

Towards a complete solution ...Problem description:I need to model the surface damage due to the impact

of laser to harden the material bulk. I need access to models including material bulk properties and interaction with intense electromagnetic fields.

Task description: I need 64 nodes of SP-2 at Argonne to run my MPI-based executable “a.out” you can find in “/tmp/users/haupt” on

marylin.npac.syr.edu. In addition, I need any idle workstationwith jdk1.1 installed. Make sure that the output of my a.out is

transferred to that workstation

Middle-Tier: map the user’s task description onto the resource specification; this may include resource discovery, and other services

Resource Specification

Resource Allocation: run, transfer data, run

6/12/99 Java Grande T. Haupt 5

Services User Modules

Back-End Resources

Front-End

Back-End services comprise Tier 3.

Distributed, object-based, scalable, and reusable Web server, Object brokerand Resource Manager Middleware forms Tier 2

Three-Tier Architecture

Tier 1 is a high-level, browser-based Front End for visual programming (including selection of applications, generation of input data sets, specification of resources, post-processing and visualizations)

Target Architecture

Middle-Tier

Resource Specification

Abstract Task Specification

CTA specificknowledgedatabases

VisualAuthoring

Tools

User andGroup

Profiles

ResourceIdentificationand Access

VisualizationsCollaboration

WebFlow

Back-End Resources

Problem Solving Environment

6/12/99 Java Grande T. Haupt 7

Portal Page

• Provides initial access to the Gateway.

• After mutual authentication of the user and the Gateway server, creates a user context, and returns a (signed) control applet.

6/12/99 Java Grande T. Haupt 8

User and Group Profile

• Controls the user/group environment– file access

– job monitoring

– ...

• Allows for customization– preferences

– users with disabilities

– ...

• History of actions

• Scientific notebook

6/12/99 Java Grande T. Haupt 9

Control Applet• The control applet is responsible for maintaining the

session, and direct communication with the middle-tier. • Direct communication is the most efficient, but since it is

buried into an applet, this mechanism is not readily customizable.

• The generic services, such as file service (upload, download, edit, copy, move, delete) and job services (show current jobs/show queues/kill jobs) will be supported this way. [combination of the user context and a query]

• The Gateway will also support a non-direct communication with the middle-tier through servelts.

6/12/99 Java Grande T. Haupt 10

Screen Dump of the Control Applet

6/12/99 Java Grande T. Haupt 11

Application Domain DataBases

PSE Example: CCM

Ken Flurchick, http://www.osc.edu/~kenf/theGateway

1. Enter the Gateway system2. Define your problem3. Identify resources (software and hardware)4. Create input file5. Run your application6. Analyze results

6/12/99 Java Grande T. Haupt 12

6/12/99 Java Grande T. Haupt 13

Selecting a Predefined Task

6/12/99 Java Grande T. Haupt 14

Visual Authoring Tools

• Allows for composition of the computational task from components (reusable modules)

• Different tools to support various programming models such as data parallel, task parallel, data flow, object oriented

• No assumption on granularity

• Metadata about components and support for archiving and mining the components

• Support for instrumentation and steering

T. Haupt 15

Example: Data Flow

6/12/99 Java Grande T. Haupt 16

Example: DARP

Example of a portalNavigate and choose an existing application

to solve the problem at hand.Import all necessary data.

Retrieve data

Pre/post-processing

Run simulations

Select host

Select model

Set parameters

Run

Processing Task Specification

XMLA visual representation

is converted into a XMLdocument

XMLservice

WebServer

save

parse

ApplContext

Generates Java code to add modules to ApplContextPublishes IOR

Front-End Applet

Middle-Tier

6/12/99 Java Grande T. Haupt 19

Other toolboxes

• Visualizations

• Collaboration

• Scientific notebook

• ...

6/12/99 Java Grande T. Haupt 20

CORBA Based Middle-Tier

Mesh of WebFlow Serversimplemented as CORBA objects

that manage and coordinate

distributed computation.

Front End

GatekeeperAuthenticationAuthorization

User 1 User 2

Application 1

Application 2

App 2App 1

WebFlow Server

WebFlow server is given by a hierarchy of containers

and components

WebFlow server hosts users and services

Each user maintainsa number of applications

composed of custom modules

and common services

WebFlow Services

T. Haupt 22

WebFlow Context HierarchyMaster Server (Gatekeeper)

Slave Server

Slave Server

User Context

Application Context

Module

Slave Server Proxy

6/12/99 Java Grande T. Haupt 23

Module functionality

• May act as a client for a back-end service such as Globus GRAM or a database

• May invoke other Gateway Middle Tier services such as File Services or Resource Discovery

• May implement the desired functionality internally (say, in Java) [thus not a proxy]

• May interact with other modules and the Front End through events

6/12/99 Java Grande T. Haupt 24

Services User Modules

Browserbased

Front-End

Middle-Tier modulesserve as proxies ofBack-End Services

Browserbased

Front-End

User Space Definition and Task Specification

Metacomputing Services

Back-End Resources

6/12/99 Java Grande T. Haupt 25

Back End Services

• Access to HPCC (via Globus)

• Access to distributed databases (via JDBC)

• Access to mass storage

• Access to the Internet resources

• Access to desktop application and local data

• Access to code repositories

6/12/99 Java Grande T. Haupt 26

WebFlow over Globus

• In order to run WebFlow over Globus there must be at least one WebFlow node capable of executing Globus commands, such as globusrun

• Jobs that require computational power of massively parallel computers are directed to the Globus domain, while other jobs can be launched on much more modest platforms, such as the user’s desktop or even a

laptop running Windows NT.

Bridge between WebFlow and Globus

SECIOP

Security Model (Keberos)

Front End Applet

SECIOP

authentication& authorization

Gatekeeper

delegation

HPCC resources

GSSAPIGSSAPI

Layer 1: secure Web

Layer 2: secure CORBA

Layer 3: Secure access to resources

Policies defined by resource owners