Date post: | 16-Jan-2016 |
Category: |
Documents |
Upload: | william-harmon |
View: | 218 times |
Download: | 0 times |
Grid MP at ISIS
Tom Griffin, ISIS Facility
Introduction
• About ISIS
• Why Grid MP?
• About Grid MP
• Examples
• The future
Introduction
• ISIS - World's brightest pulsed neutron and muon
source• Running since 1984; 1600 user base; ~200 staff• Neutrons and muons for condensed matter science
• Based at the Rutherford Appleton Laboratory
• Driven by scientific output, not computer science
Why does ISIS need faster processing?• Many compute intensive programs and problems
• Many applications which could be compute
intensive
• Large range of application domains
• Simulations
• Analysing results
• Designing new instruments
• Visualising ideas
Why Grid MP?
• Neutron science not computer science
• Run by data analysis group
• Long interest in distributed computing
• Sun Workstations
• Condor
• Entropia DC Grid
• United Devices
Why Grid MP?
• ‘Out of box’ solution
• United Devices Grid MP is the ‘market leader’
• Customers include:
• Johnson & Johnson
• GlaxoSmithKline
• Novartis
• National Physical Laboratory
Why Grid MP?
• Set up and running real applications in under 3
hours
• Excellent service level
• Get what you pay for!
• Queries and problems always answered promptly
• Active development
The United Devices System• Server hardware
• We use two, dual Xeon servers + 280 client licenses• Could (will) easily cope with more clients
• Software• Servers run RedHat Linux Advanced Server / DB2• Clients available for Windows, Linux, SPARCs and Macs
•Programming• MGSI – Web Services interface – XML, SOAP• Accessed with C++ and Java classes etc
• Management Console• Web browser based• Can manage services, jobs, devices etc
Installing and Deploying the System• Servers
• Complete set up in under 3 hours
• Virtually self maintaining
• Clients• Windows only so far
• MSI Installer
• approx 20 seconds
• SMS
• MP Agent User
• Install to other OSs looks straightforward
Grid MP Mode of operation• Client – Server
• Submitting PC breaks up Job
• Uploads data to server
• Programs reside on the server
• Server distributes Workunits to clients
• Clients return results to server
• User downloads results from server
Getting an Application Running
• Don’t touch executable code
• Wrap and upload executable
• Write ‘Application Service’
• Typically only a few hours work
Some Features of the Grid MP System• Device to Job matching
• Jobs have requirements – CPU, RAM, free disk space etc
• Programs have requirements
• Devices have preferences
• choose what programs they want to run
• Administrator can choose which users may submit to which
machines
Some Features of the Grid MP System• Timing options
• When to execute
• When to communicate
Some Features of the Grid MP System• Good access/privileges control
• Very powerful
• Read, update, create, delete
• Can hook into Active Directory, Kerberos etc
• Built in basic reporting
• Job / Program CPU hours
• Performance stats
What do we use it for?
• 16 Applications running so far:
• Several molecular dynamics apps
• several neutron instrument simulation packages
• crystal structure solution
• maximum entropy calculations
• POVRay image rendering
What performance do we get?
• Roughly linear for current applications
• Running on 100 machines is 100 times faster
• Average CPU 2.03Ghz
• High percentage of AMD Athlons
Web Interface - Monitoring Jobs
Problems Encountered
• Mercifully few
• Only once the Grid’s fault
• Memory thresholding
• Odd program interactions
• Fortran and OpenGL
• Easiest thing to blame
• Too abstract for some users (no big box)• Stealing my cycles
• Expansion leads to political problems
‘Social’ Issues
• Expansion• Applied for 400 licenses (£60k)
• Change in licensing model
The Future
• Bottom line• Unlimited licenses
• Total $250K
• No technical issues
• Plenty of political issues
• Single sign-on