Grid Computing 3
Outline
• Conception
• Basic Issues
• Grid Architecture
• Standards for Grid Environments
• Key Components
• Applications
The Grid dream is to allow users to tap into resources off the Internet as easily as electrical power can be
drawn from a wall socket
Grid Dream
4 Grid Computing
Compute Grids
Parallelize logic execution
Data Grids
Parallelize data storage
Grid Computing =
Data Grids + Compute Grids
Data Partitioning + Affinity Map/Reduce
What is Grid Computing?
5 Grid Computing
Grid Computing 6
What is Grid Computing?
• Allows sharing and coordinated use of diverse resources in dynamic, distributed “virtual organizations”.
Why Grid Computing?
The idea of Grid computing resulted from the confluence of three developments:
The proliferation of largely unused computing resources (especially desktop computers)
Their greatly increased CPU speed in recent years
The widespread availability of fast, universal network connections (the Internet).77
7 Grid Computing
High performance computers (formerly called supercomputers) are very expensive to buy and maintain.
Much of the enhancement of computing power recently has come through the application of multiple cpus to a problem (e.g., NCSC had a 720 processor IBM parallel computer).
Many computing tasks relegated to these (especially massively parallel) computers could be performed by a ―divide and conquer‖ strategy using many more, although slower, processors as are available on a Grid.
Why Grid Computing?
8 Grid Computing
Grid Computing 10
Electrical Power Grid Analogy
Electrical power grid
• users (or electrical appliances) get access to electricity through wall sockets with no care or consideration for where or how the electricity is actually generated.
• “The power grid” links together power plants of many different kinds
The Grid • users (or client applications) gain
access to computing resources (processors, storage, data, applications, and so on) as needed with little or no knowledge of where those resources are located or what the underlying technologies, hardware, operating system, and so on are
• "the Grid" links together computing resources (PCs, workstations, servers, storage elements) and provides the mechanism needed to access them.
Grid Computing 11
Why need Grid Computing?
• Core networking technology now accelerates at a much faster rate than advances in microprocessor
speeds • Exploiting under utilized resources
• Parallel CPU capacity
• Virtual resources and virtual organizations for collaboration
• Access to additional resources
Grid Computing 12
Who needs Grid Computing?
• Not just computer scientists…
• scientists “hit the wall” when faced with situations: – The amount of data they need is huge and the data is
stored in different institutions.
– The amount of similar calculations the scientist has to do is huge.
• Other areas: – Government
– Business
– Education
– Industrial design
– ……
Grid Computing 13
Types of resources
• Computation
• Storage
• Communications
• Software and licenses
• Special equipment, capacities, architectures, and policies
Grid Computing 14
Job Scheduling
Grid Computing 15
Security
• Access policy - What is shared? Who is allowed to share? When can sharing occur?
• Authentication - How do you identify a user or resource?
• Authorization -How do you determine whether a certain operation is consistent with the rules?
Grid Computing 16
Grid Security Model
Grid Computing 17
Grid User Roles ---A User’s Perspective
• Enrolling and installing grid software
• Logging onto the grid
• Queries and submitting jobs
• Data configuration
• Monitoring progress and recovery
• Reserving resources
Grid Computing 18
Grid User Roles ---An Administrator’s Perspective
• Planning
• Installation
• Managing enrollment of donors and users
• Certificate authority
• Resource management
• Data sharing
Grid Computing 19
Grid Architecture
Application
Collective
Resource
Connectivity
Fabric
Application
Transport
Internet
Link
GRID Internet
Grid Computing 20
Grid Architecture
• Fabric layer: Provides the resources to which shared access is mediated by Grid protocols.
• Connectivity layer: Defines the core communication and authentication protocols required for grid-specific network functions.
• Resource layer: Defines protocols, APIs, and SDKs for secure negotiations, initiation, monitoring control, accounting and payment of sharing operations on individual resources.
• Collective Layer: Contains protocols and services that capture interactions among a collection of resources.
• Application Layer: These are user applications that operate within VO environment.
Grid Computing 21
Key Components
• Portal/user interface
Grid Computing 22
Key Components
• Security
–Grid Security Infrastructure (GSI)
Grid Computing 23
Key Components
• Broker
–Monitoring and Discovery Service (MDS)
Grid Computing 24
Key Components
• Scheduler
Grid Computing 25
Key Components
• Data management
– Grid Access to Secondary Storage (GASS)
Grid Computing 26
Key Components
• Job and resource management
– Grid Resource Allocation Manager (GRAM)
Grid Computing 27
Applications • The Southern California Earthquake Center uses
Globus software to visualize earthquake simulation data.
Grid Computing 28
Applications • Scientists in the Earth System Grid (ESG) are producing,
archiving, and providing access to climate data that advances our understanding of global climate change. ESG uses Globus software for security, data movement, and
system monitoring.
Grid Computing 29
Applications
• Computational scientists at Brown University are using the Globus Toolkit and MPICH-G2 to simulate the flow of blood through human arteries.
Grid Computing 30
Applications • Globus Toolkit-driven Grid computing is central to
management of large datasets generated by colliders such as those at CERN. This simulation shows two colliding lead ions just after impact, with quarks in red, blue, and green and hadrons in white.
Grid Computing 31
Software implementations and middleware
• Advanced Resource Connector (NorduGrid's ARC)
• Berkeley Open Infrastructure for Network Computing (BOINC)
• Globus Toolkit
• Sun Grid Engine
• ProActive
• UNICORE
• SDSC Storage resource broker (data grid)
• gLite (EGEE)
• NInf GridRPC
• IceGrid
• Java CoG Kit
• Alchemi
• GridGain
• gridGISTICS
• Gridbus Middleware
• Vishwa
• UGP
• GRIA
Grid Computing 32
Organizations & Production Grids
Alliances and organizations
• Open Grid Forum (Formerly Global Grid Forum)
• Object Management Group
Production grids
• Enabling Grids for E-sciencE
• NorduGrid
• Open Science Grid
• OurGrid
• Sun Grid
• Xgrid
• Distributed European Infrastructure for Supercomputing Applications DEISA
• FusionGrid
Grid Computing 33
Grid Projects International Grid Projects • Open Middleware Infrastructure Institute Europe (OMII-Europe) -
May 2006 -> May 2008
• Enabling Grids for E-sciencE (EGEE) - March 2004 -> March 2006
• Enabling Grids for E-sciencE II (EGEE II) - April 2006 -> April 2008
• BREIN — September 2007 → August 2009
• DataTAG - January 2001 -> January 2003
• European DataGrid (EDG) - March 2001 -> March 2004
• BalticGrid - November 2005 -> April 2008
National Grid Projects • D-Grid (German)
• GARUDA (Indian)
• grid computing project at VECC (Calcutta, India)
• China Grid Project
• INFN Grid (Italian)
• KnowledgeGrid Malaysia
• NAREGI Project
• Singapore National Grid Project
• Thai National Grid Project
• BELNET Grid, Belgium
Grid Computing 34
Prospect of Grid computing
The Grid aims ultimately to turn the global network of computers into one vast
computational resource.
Related to many areas in computer science
Being developed by hundreds of researchers and software engineers around the world.
The goal of IRAN-GRID project is to provide: uniform, integrated computational
platform/infrastructure based on the existing computational clusters located at major Universities and Research
establishments.
During the first phase of this project: 12 Universities and Research establishments connected via IPM-GCG clusters.
What is Iran National Grid?
35 Grid Computing
Grid Computing 36
مامریت
داوشگا ي مراکس تحقیقاتی در سطح کشر برای تی 12اتصال • تان محاسباتی باال
:استان تران پژطگا داص اي بيادي• داطگا صعتي اهيركبير• داطگا صعتي ضريف• داطگا تراى• داطگا علن صعت ايراى• داطگا تربيت هدرس•
:بروام آتی داطگا ضيد بطتي• داطگا ضاد• پژطگا افضا• پژطگا ضيخ بايي• هسس اقياس ضاسي•
:استان اصفان داطگا صعتي اصفاى•
:بروام آتی داطگا اصفاى•
:استان فارس داطگا صعتي ضيراز• داطگا ضيراز•
:استان خراسان رضی داطگا فردسي هطد•
:استان مرکسی داطگا تفرش •
:استان قسيیه داطگا بيي الوللي اهام خويي•
:استان کرمان پارك علن في آري كرهاى •
:استان کرماوشا داطگا رازي كرهاطا•
:بروام آتی
:استان زوجان داطگا زجاى•
:استان آذربایجان شرقی داطگا تبريس•
:استان مدان داطگا بعلي سيا•
:استان سمىان داطگا ضارد•
:استان سیستان ي بلچستان داطگا سيستاى بلچستاى•
نقاط متصل به تورين
تي تورين برنامه آ
EGI
This platform will enable and support e-Science in Iran by offering:
high performance computing (HPC)
high-throughput computing (HTC)
data storage capacity needed for very large scale computational projects
What is Iran National Grid?
39 Grid Computing
User Interface (UI) [ui.ipm.ac.ir]
Resource Broker or Workload Management Service (RB, WMS) [wms.ipm.ac.ir]
Information System (top-bdii) [top-bdii.ipm.ac.ir]
Virtual Organization Membership Server (VOMS) [voms.ipm.ac.ir]
Logical File Catalog (LFC) [lfc.ipm.ac.ir]
Monitoring and accounting [monbox.ipm.ac.ir]
IPM-grid Global-Level Architecture
40
The term "grid computing" suggests a computing paradigm similar to an electric power grid - a variety of resources contribute power into a shared "pool" for many consumers to access on an as-needed basis
Ideally the user does not know or care where the computing operation is being performed; the process is invisible to the user.
Middleware handles security, authentication, authorization, resource selection and routing of input and output seamlessly.
How to use Grid?
41
CERN
SETI@home
DNet (distributed.net)
GRID.ORG (anti-cancer ligand screening)
IBM Smallpox cure
Entropia.org
…
Examples of Grid Projects
42
Registration
43
Registration
44
In grid computing a process is called a job.
Users submit jobs to the system via one of the participating institutions
A user of Grid computing does not need to have the data and the software on the same computer, and neither must be on the user’s home (login) computer.
Job Management
45
There are 3 prerequisites for starting job submission:
Prerequisites
Receiving Your
Certificate
Signing for a VO
Obtaining Access to
UI
Receiving Your
Certificate
Signing for a VO
Obtaining Access to
UI
46
Connecting to the UI
Installing the
certificate
Creating a proxy
Running a job
Job Submission
Connecting to the UI
Installing the
certificate
Creating a proxy
Nightingale$ ssh [email protected] -p 4422
$mkdir .globus $mv YourCert .p12 .globus/ $cd .globus/ $openssl pkcs12 –nocerts –in *.p12 –out userkey.pem $openssl pkcs12 –clcerts –nokeys –in *.p12 -out usercert.pem
$voms-proxy-init -voms ipmgrid $voms-proxy-info –all
47
Job Submission
Connecting to the UI
Installing the
certificate
Creating a proxy
Running a job
Running a job
Type="Job"; Executable = "/bin/hostname"; StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.out","std.err"};
$ glite-wms-job-submit -a -r Resource hostname.jdl $ glite-wms-job-status MyJobID $ glite-wms-job-output - - dir output MyJobId
48
grid.ipm.ir
Grid Computing 49
grid.ipm.ir
Grid Computing 50
grid.ipm.ir
Grid Computing 51
grid.ipm.ir
Grid Computing 52
Grid Computing 53
Reference • ―The Anatomy of the Grid(Enabling Scalable Virtual
Organizations)‖
---by Ian Foster, Carl Kesselman, Steven Tuecke
• ―Physiology of the Grid (An Open Grid Services Architecture for Distributed Systems Integration )‖
---by Ian Foster, Carl Kesselman, Jeffrey M. Nick, Steven Tuecke
• ―Grid Computing:Past,Present, and Future‖--- by Elias Kourpas, June 2006
• ―Introduction to Grid Computing‖ ---IBM Redbook,2005
• IBM Grid Computing:www-03.ibm.com/grid/index.shtml
• Globus website: www.globus.org
54
Thank You
Question?