OpenPOWER Roadmap towards CORALKlausGottschalkHPCArchitectPOWERPlatform
HPCAdvisoryCouncilMeetingLugano,March23rd,2016
FundamentalforcesareacceleratingchangeinourindustryProcessortechnologyalonecannolongerprovidethepriceperformancegainsnecessary tosustain
Moore’sLaw
Scale-out & Hyperscale Data Centers
Hybrid Cloud
Open Solutions
ITconsumptionmodelsareexpandingto:
Abetterprocessorandfullsystemstackopeninnovationrequired
Price
/Perform
ance
Moore’sLaw
ProcessorTechnology
2000 2020
Firmware/OSAcceleratorsSoftwareStorageNetwork
2
• Data&infrastructuresilos
• Datainducedbottlenecks
• Blindlyaddingcapacity
• Dataduplicationandmoresilos
• Adhoccloudusage
Complexity
Cost
Availability
Traditionalcompute&storageapproacheswon’tsuffice…
3
Price
/Perform
ance Semiconductor
TechnologyandProcessors
2000 2020
IBMHPCstrategy
POWER8era
ShiftbacktowardstheMoore’sLawpredictionthrough:1. IBMHPCInnovation(processorarchitectureenhancement,scalablefilesystems,workflowmanagement)2. Acceleration throughpartnerecosystem(e.g.NVIDIAGPUsdeliver2Xperf/wattand~3Xperformance)
POWER9 era
IBMHPCInnovation
Acceleration
Moore’sLaw,ClientExpectations
4
IBMVision:Data-CentricComputing
• Hardwareandsoftwaretosupport&enablecomputeindata• AllowworkloadstorunwheretheyrunbestMinimizeDataMotion
• Introduce“active”systemelementsincludingnetwork,memory,storage
• Globalaccesstoshareddata
EnableComputeEverywhere
• Balanced,composable architecture• Modularandupgradeabledesignscalableupto100sofracksModularity
• Userealworkloads/workflowstodrivedesignpoints• Co-designforcustomervalue
Application-drivendesign
5
IBMHPCInnovationsandInvestments:OurCommitmenttoHPC
ProcessorDesign• POWER8,POWER9,14nmProcessorDesign
ProcessorInterfaces• CAPI,POWERwithNVLinkTechnology
Filesystems&StorageSolutions• IBMSpectrumScale• ElasticStorageServer• HPSS• IBMFlashSystem
Compilers,Development,Libraries• IBMXL,Parallel Environment,ESSL
MPIandClusterProgramming• IBMPlatformMPI,OpenMP 3.1and4.0Development
Workload&ClusterManagement• IBMPlatformClusterManager,xCAT
• IBM PlatformLSFPowerandData-AwareScheduling
6
Highlights:IBMHPCaroundtheWorld2014-2015
IBM,Mellanox,andNVIDIAawarded$325MU.S.DepartmentofEnergy’sCORALSupercomputers
GENCIannouncescollaboration withIBMon
speedingthepathtoexascale computing.
DESYcollaborateswithIBMResearchselectsSoftwareDefinedStorageSDI,IBM
Powerforparticleacceleratordata
UKGovernmentInvests£115Million inBigData&CognitiveComputing
ResearchwithSTFCandIBM
LeadingoilandgasfirmselectsIBMforworkload
management
LSUselectsacomprehensiveIBMHPCsolutionforgenomicsresearch
IntegratedPortfolio:IBMPowerSystemsSoftwareDefinedInfrastructureIBMStorageIBMResearch
Withmany,manymorearoundtheworld… 7
OpenPOWER:OpenArchitectureforHPC &BigData
ProcessorIPLicensing
OpenInterfaces
Systems&Software
LicensingIPtoenablesemiconductorpartnerslikeSuzhouPowercore tobuildPOWERchips
TightintegrationusingCAPI&NVLink withAccelerators(NVIDIA,Xilinx,Altera),Networking(Mellanox),Storage(CAPIFlash)
EnablingInnovativePOWER-basedserversfromPartners&OpenCompute andSharingOpenSourceSoftwareincludingFirmware&Hypervisor
8
MajorWinsEndorsetheStrategyandEstablishthePathForward
IBM,Mellanox,andNVIDIAawarded$325M
U.S.DepartmentofEnergy’sCORALSupercomputers
IBM&UK’sSTFC PartnerforBigData&CognitiveComputing Researchin£313MPartnership
CORAL: Leadership Class Supercomputers 5X – 10X HIGHER APP PERF THAN CURRENT SYSTEMS
US,UK,GermanScientificResearchCommunitiesSelectOpenPOWERorCommittoOpenPOWERCollaboration
PADCOpeningOctober12th,2015
Jülich Supercomputing CentrePOWERAccelerationandDesignCenter(PADC) RoadmapTowardCORAL
1900+ApplicationsRunonPOWERBigData&
MachineLearningCloud MobileEnterprise
Major Linux Distros
HPC
miniDFTCTHBLASTBowtieBWAFASTAHMMERGATKSOAP3STAC-A2SHOC
Graph500Ilog
CHARMMGROMACSNAMDAMBERRTM
GAMESSWRF
HYCOMHOMME
LESMiniGhostAMG2013OpenFOAM
11
POWER8:ProcessorPerformanceLeadership
MemoryBuffer
DRAMChips
POWER812Cores,96Threads4LevelLargeCachesUpto1TBpersocket
Upto230GB/ssustained
FasterCores8ThreadsPerCore
LargerCachesDirectAccelerator
Interconnect
3xHigherMemoryBandwidth,1TB
MemoryperSocket
12
HPCApplicationPerformance
0,0
0,5
1,0
1,5
2,0
2,5
3,0Re
lativ
ePe
rforman
ce
HaswellBased POWER8– S822LC
POWER8:Upto2.5xFasteronApplicationsoverIntelHaswellx86CPUs
13
SharedSystemMemoryviaCoherentInterface 2.5xFasterCPU-GPUConnectionviaNVLink
OpenPOWER:TightintegrationusingCAPI&NVLink
POWERCPUs
Memory
FPGA
CAPICoherentAcceleratorProcessorInterface
14
IBMOpenPOWER-basedHPC Roadmap
2015 2016 2017
POWER8POWER8+
POWER9
OpenPowerCAPI Interface
NVLink
EnhancedCAPI &NVLink
Connect-IBFDRInfinibandPCIe Gen3
ConnectX-4EDR Infiniband
CAPI overPCIe Gen3
ConnectX-5Next-GenInfiniband
EnhancedCAPI overPCIe Gen4
MellanoxInterconnectTechnology
IBMCPUs
NVIDIAGPUs KeplerPCIe Gen3
VoltaEnhancedNVLink
PascalNVLink
IBMNodes15
NewIBM“LC”PowerSystemsforLinux
812LCCPU
Memory
1xPOWER8CPU10cores,2.9-3.3GHz
Upto1TeraByteMemory115GB/sMemoryBandwith14Drives(84TB,HDD,SSD)
OptimizedforHadoop,Spark,In-MemoryAnalytics
822LCCPU
Memory
OptimizedforDatabasesandCloudWorkloads
CPU2xPOWER8CPU
10coreseach,2.9-3.3GHzUpto1TeraByteMemory
230GB/sMemoryBandwith2Drives(2TB,HDD,SSD)
822LCHPC
CPU
MemoryBuiltforHPC andDeep
Learning
CPU2xPOWER8CPU
10coreseach,2.9-3.3GHzUpto1TeraByteMemory
230GB/sMemoryBandwith2xNVIDIATeslaK80GPUs
GPU GPU
16
IBMSoftwareDefinedInfrastructurefordiverseworkloads
IBMPlatformComputing
BigData/Hadoop
HighPerformanceAnalytics
HighPerfComputing
Makelotsofcomputers&storagelooklikeoneresourcepool
ScaleUpandOut
Prioritizedmatchingofsupplywithdemand
Benefitsü Efficientutilizationü HighAvailabilityü HighThroughputü HighPerformanceü SLAPrioritizationü Reducedcost
IBMSpectrumStorage
x86 Linuxonz
IBMFlashSystem
IBMHybridStorage IBMTapeStorage
On-premise,On-cloud,HybridInfrastructure(heterogeneous distributedcomputing andstorageenvironment)
OtherCommercial
Other
NewGenApp
Framework
Other
17
IBMPlatformLSF10.1(Q22016)
• Productivity-centric, enterprise-class workload management for HPC environments
• Intelligent policy-based data management• Optimized for both high throughput and traditional
HPC applications
12xlesswork
150xlesswork
6xlesswork
LSF9.1.2(2013) LSF9.1.3.3(2015)5.8million jobs/hr
Highthroughputsyntheticbenchmarkexternallyauditedandverified
Superiorrepeatableperformance=betterbusinessoutcomes
IBMConfidentialuntilannounced
EaseofUse•Understandwhyworkloadisnotrunning
•Understandwhenworkwillrun
HPCAccessibility•Webenabledworkflows•Simplifymigrationtothecloud
PerformanceatScale
• Delivering nextevolution ofperformanceandscalability
18
Wellcome TrustSangerInstituteUsinganalyticstooptimizehighperformancecomputingcapacityacross15,000cores
ProblemThe Wellcome Trust Sanger Institute, a non-profit research organization based in Hinxton, England, wanted to accelerate results to meet scientific journal publication deadlines and secure funding for future projects.
Solution• Platform LSF® policy-based scheduler• IBM® Platform™ Analytics with Custom dashboards showing cluster
utilization according to criteria such as job, job submitter & memory use• IBM® Platform™ Process Manager for workflow design and executionOutcome• Each job runs on the optimal compute node increasing throughput across
the HPC environment• Researchers understand the factors that reduce processing efficiency,
and take action to improve performance• A clear record of efficient compute utilization enables a
strong business case for future growth investments
Industry: ResearchProducts: IBMPlatform™LSF®
IBMPlatformAnalyticsIBMPlatformProcessManager
Overview
Profile
“AnalyticsisoneofthemostsignificantITinvestmentsthatwehavemade.WeneededtheIBMsolution todelivertheHPCutilizationrequiredtohelpourresearchteamsgenerateresultsrapidly,meettheirpublicationdeadlinesand,ultimately,securenewfunding.”
Dr.PeterClaphamPrincipalSystemsAdministrator,InformaticsSystemsGroup,Wellcome TrustSangerInstitute
19
IBMSpectrumScale4.2andIBMElasticStorageServer
• Reducecapitalandoperatingexpenseforstoragesupportingverylargescaledata• Enableacommonpoolofstorageformodernworkloadswithunifiedfileandobjectstorage• Policy-drivencompressionandQualitiesofService
• Accelerateanalyticsapplications• EliminateneedtomovedatabackandforthfromHadoop/HDFSstoragesiloswithintegratedHDFS
• AutomateBigDatadeploymentwithApacheAmbari support
• Speedandsimplifydeployment• NewSpectrumScaleGUIconsistentwithSpectrumStoragefamily• ExpandmanagementoptionswithSpectrumControlintegration• Fullyfunctionalvirtualmachinedemonstrationlowersbarrierstoadoption
AvailablesinceNovember20(IBMSpectrumScale)November6(IBMElasticStorageServer)
20
DESY- ResearchinstituteinGermanyAcceleratingexperimentaloutcomeswithIBMSoftwareDefinedInfrastructure
Solution components
The Challenge Deutsches Elektronen-Synchrotron needed a solution that could keep up with one of the most powerful x-rays in the world, rapidly delivering experimental results to researchers
The SolutionA high-performance IBM storage and compute solution to support increasingly sophisticated experiments and offers analysis-as-a-service, allowing scientists to tap into DESY’s particle accelerator facilities and access the data from anywhere in the world.
The Results§ Accelerates research with data available for analysis in
4 minutes, down from 2-3 days§ Accommodates more demanding projects, handling 20GB/second§ Scales up for continuing innovation
• IBM® Elastic Storage™ Server• IBM Spectrum Scale• IBM Platform HPC• IBM Power Systems™ running Linux• IBM Advanced Business Partner:
MCS Moorbek Computer Systeme GmbH
"The ability to provide scientists with data on very short time scales has changed the way experiments are done at the beam line.”--Steve Aplin, senior scientist 21
ActionableIntelligencewithOptimizedPerformanceIBMFlashSystemandSpectrumScale
IBMFlashSystem poweredbyFlashCore™Technologyisoptimizedtodeliverextremeperformance,reliability,andlowlatencyinthemostdemandingenvironments
IBM FlashSystem
All active files
Eliminate costly performance bottlenecks
IBMFlashSystem
Hot Files
HDD Storage All other files
Data Metadata
IBM FlashSystemSpectrum Scale
Primary storage
1MillionIOPS
ResponseTimesunder0.5millisecond
Over6GB/ssequential
readthroughput
Asacachedevice Asastoragetier Asametadatastoragedevice
22
LeadingObjectStorageInnovation:SpectrumScaleandCleversafe
IBMofferscomplementaryObjectstorageofferings- eachdeliveringindustryleadinginnovationsfordifferentworkloads/usecases
• Cleversafe isdesignedtoenablesecure,largescaleactivearchivesandcloud-basedcontentrepositorieswherefocusisonlowestcostatscale• Erasurecodingdeliverscost-efficientreliability,requiringlessspacethantriplecopyapproaches• UniquekeylessencryptiontechnologyprovidesstrongsecurityforObjectarchives
• SpectrumScaleofferslargescaleFile& Objectstoragethatsupportshighperformanceprocessingandbigdataanalytics• Supportsmultiplearchitectures:storagerichservers;NASclusters;globallydistributedgrids• POSIXsupportforscale-outOLTPdatabasesanddatawarehouses• IntegratedlifecyclemanagementacrossFlash,DiskandTapetiers
23
HighPerformanceStorageSystem(HPSS)
ImprovedstorageeconomicsanddataavailabilityOpenStack Swift
• OakRidgeNationalLaboratorycutredundanttapecostsby75%with4+PHPSSRAITandupto872MB/sfile-transfers
• Enterprisetaperecommendedaccessorder(RAO)forfastertaperecalls
• HPSSTrashCan
• Filesystemandtraditionaldiskstoragetiers• ProductionreadyonPower8
• OpenStack “SwiftOnHPSS”providesobjectinterfaceforfilesharing
• Stopcorruptedfilesfrommakingittotapewithend-to-enddataintegrity
• 219MB/spertapedriveusing2.6MBfiles
24
IBMishelpingourclients…
ü Expandfromtraditionalcompute-centrictoadata-centricHPCmodelü Efficientlymanagethelifecycleofalldataincludingfilesandobjectsü Achievenewinsightsfromalldatawhileimprovingdataeconomicsü Store,analyzeandprotectdatawithahyperscale convergedinfrastructure
MaximizePerformance
EnableNewWorkloads
OptimizeEfficiency
25
MoreInformationFormoreinformation:
• Highperformancecomputing(HPC)andtechnicalcomputingsolutions• http://www-03.ibm.com/systems/power/solutions/high-performance-computing/
• http://www-03.ibm.com/systems/technicalcomputing/
• FindIT:yourgatewaytofindandusethetopassetsyouneedforanygivensalessituation• https://findit.cloud.dst.ibm.com
• Redbook:ImplementinganIBMHigh-PerformanceComputingSolutionon
IBMPOWER8• http://www.redbooks.ibm.com/abstracts/sg248263.html?Open=
26