Date post: | 12-Apr-2017 |
Category: |
Software |
Upload: | jeff-reser |
View: | 105 times |
Download: | 0 times |
Keeping Pace with Growing High Performance Computing Needs
SUSE Linux Enterprise Server for High
Performance Computing
Jeff Reser
Simplifying lives with a highly scalable, high
performance open source operating system
that harnesses the power of the
supercomputer for your day-to-day business
Solving computational, data-intensive, or
numerically-intensive tasks
Reducing the time and effort required to set
up and maintain HPC clusters
Ensuring that all components of the HPC
stack work together
© Copyright 2016 SUSE LLC. All rights reserved.
2
SUSE Linux Enterprise Server for High Performance
Computing 12 SP2
What’s New in SLES for HPC
3
• Realize faster time to value with HPC Module packaging that includes leading-edge HPC components and enables easier selling, simplified maintenance and more focus – with affordable pricing
• Accelerate innovation with added support for the Ceph File System support, enabling you to transform your enterprise storage infrastructure to reduce costs while providing unlimited scalability
• Now commercially available as part of Intel HPC Orchestrator –
Intel used SLES plus a number of OpenHPC community
components
• Microsoft Azure Test Drive program includes SLES for HPC for a free trial
• ARM64 partnering with OpenHPC, Cavium, ARM, Applied Micro
• Increase efficiency with Intel Omni-Path Architecture (OPA)
support for performance and throughput
Top HPC Challenges and Needs
Control
Infrastructure
Optimize
Operations
Increase
Performance
System performance needs to be increased
amid IT budgets and staff reductions
HPC is business-critical, but managed by a
separate department with a scientific flavor
Need support and interoperability across the
entire HPC stack, without dependencies on a
single hardware vendor
5© Copyright 2016 SUSE LLC. All rights reserved.
Top HPC Market Challenges
HPC market still developing
Stack components provided by various vendors
Applications need to be updated to leverage HPC
Mix of small and big vendors
Segmented into commercial and scientific
6© Copyright 2016 SUSE LLC. All rights reserved.
Linux on 99.4% of the Top 500 Supercomputers
Linux
Unix
Mixed
Windows
BSD
Mac
70%
60%
50%
40%
30%
20%
10%
0%
80%
90%
100%1
99
4
TO
DA
Y
8
Linux in HPCHalf of the world’s 20 largest supercomputers run on SUSE
50%
© Copyright 2016 SUSE LLC. All rights reserved.
13.4% SUSE OS Share, including Cray Linux Environment
Growing HPC Adoption
9
Universities
Government
Academia & Research Institutions
Financial Services: Risk Analysis
Oil & Gas Exploration, Production
Semiconductor: Electronics
Design Automation
Life Sciences: Pharmaceutical &
Biomedical
Manufacturing: Aerospace &
Automotive
Departmental and
workgroup compute
clusters
“High Productivity
Computing”
YESTERDAY TODAY TOMORROW
© Copyright 2016 SUSE LLC. All rights reserved.
Worldwide HPC Server Market (according to IDC)
10
CAGR 2015-2019:
• 10.1% Supercomputer (>$500K)
• 0.7% Divisional ($250K-$500K)
• 9.5% Departmental ($100K-$250K)
• 8.7% Workgroup (<$100K)
• HPC still expected to be a strong
growth market
• Growing recognition of HPC’s strategic
value is helping to drive high-end sales
• Low-end buyers are back into a growth
mode
• Recognition of HPC’s strategic and
economic value will drive the exascale
race
• The formative HPDA (High Performance
Data Analysis) market will expand
opportunities for vendors (plus
machine learning, cognitive and IoT)
OpenHPC Open Source Communityopenhpc.community
12
• Linux Foundation project – SUSE is a
founding member (now 30+ full members)
• Provides common platform – standard HPC
stack – for collaboration and innovation
• Provides full HPC stack (~300 packages) on
top of SLES
• Build with SUSE build service:
http://build.openhpc.community
• Simplifies installation, configuration, and
maintenance of a custom software stack
© Copyright 2016 SUSE LLC. All rights reserved.
Partnering with Intel
14
Intel distributes SUSE Linux Enterprise Server for
High Performance Computing as part of the Intel HPC
Orchestrator
SUSE is the commercial Linux supported by the Intel
HPC stack
• Delivers a fully supported stack for HPC workloads
• Accelerates research/innovation and time to insight
• Aligns new optimized components driven by the
broader openHPC.community
• Simplifies ongoing maintenance and support
14© Copyright 2016 SUSE LLC. All rights reserved.
https://www.suse.com/promo/intel-suse-hpc-stack
SUSE Joins Microsoft Enterprise Cloud
Alliance and Test Drive Program
15
SUSE is the only supported enterprise
Linux HPC workload on Microsoft Azure
public cloud
• Cloud-based SLES, including
automated updates, to enterprise
customers using the Microsoft Azure
cloud platform
• SLES for HPC is part of the Microsoft
Azure Test Drive program, freely
available via Azure Marketplace
15© Copyright 2016 SUSE LLC. All rights reserved.
https://www.suse.com/promo/cloud/public/azure
17
MULTI-CORE
PROCESSOR
SUPPORT
TECHNOLOGY
HIGH
PRODUCTIVITY
COMPUTING
ACADEMIC AND
RESEARCH
COOPERATION
SUSE
Highlights of High Performance Computing
17© Copyright 2016 SUSE LLC. All rights reserved.
Intel, AMD,
POWER, ARM
Kernel,
Lustre,
Ceph
OpenHPC,
Intel, IBM, SGI,
HPE, Dell, Bull,
NEC, Cray,
Cisco, Fujitsu
Total Oil &
Gas
LRZ/SuperMUC,
BSC/MareNostrum,
Tokyo Institute of
Technology/TSUBAME,
Beijing Computing
Center, NASA/Pleiades
HPC Impacts Our Lives Through Space Research
© Copyright 2016 SUSE LLC. All rights reserved.
Understanding how solar activity affects us on Earth
• Pleiades supercomputer simulations help explain NASA’s
IRIS (Interface Region Imaging Spectrograph) solar
observatory findings
• Combined with high-performance computer modeling,
simulation, and analysis back on Earth, scientists hope to
increase their understanding of solar dynamics
• Solar simulations through numerical modeling on
Pleiades, located at the NASA Advanced
Supercomputing (NAS) facility at Ames Research Center
(runs SUSE Linux Enterprise)
18
HPC Impacts Our Lives With Better Preparation for Seismic Events
© Copyright 2016 SUSE LLC. All rights reserved.
Understanding the effects of seismic activity
• Geophysicists use earthquake simulation software to
investigate seismic waves beneath the Earth’s surface
• Goal is to simulate earthquakes as accurately as
possible to be better prepared for future events
• Calculations involved in this kind of simulation are so
complex that they push even supercomputers to their
limits
• Using SuperMUC, they simulated the vibrations inside
the geometrically complex Merapi volcano on the
island of Java
19
HPC Impacts Our Lives Through Weather & Climate Forecasting
© Copyright 2016 SUSE LLC. All rights reserved.
From saving lives to just planning the weekend
• National Center for Atmospheric Research
(NCAR) has a new system – Cheyenne, running
SUSE Linux – for advancing atmospheric and
Earth science
• Helps scientists improve predictions of a range of
phenomena, from hour-by-hour risks associated
with thunderstorm outbreaks to the timing of the
11-year solar cycle and its potential impacts
• Cheyenne will be a critical tool for researchers
studying climate change, severe weather,
geomagnetic storms, seismic activity, air quality,
wildfires, and other important geoscience topics
20
HPC Impacts Our Lives Through Oil and Gas Exploration
© Copyright 2016 SUSE LLC. All rights reserved.
From getting around with our cars to just
staying at home in front of a gas fireplace
• HPC accelerates oil and gas exploration with
processing of seismic data and simulation of
deposit fluids
• As new oil and gas reserves become more
elusive, companies like Total rely increasingly
on HPC to find opportunities in an ocean of
seismic data.
• With SUSE Linux Enterprise Server, Total
enjoys the optimal combination of performance,
price and efficiency.
21
HPC Impacts Our Lives Through Media and Entertainment
22
From action-packed cinematic special
effects to games
• SUSE Linux Enterprise runs on
supercomputers in creating special effects
in movies and video games
• All with demanding requirements for ultra-
high computing for an amazing user and
viewer experience
• Linux is the OS of choice for Hollywood
animation and special effects departments,
used in production of more than 30
blockbuster films
© Copyright 2016 SUSE LLC. All rights reserved.
HPC Impacts Our Lives Through Social Phenomena Analysis
23
Used for numerous and varied purposes, from
medicine to earthquake warnings, to social
phenomenon analysis
• TSUBAME touted as the “supercomputer for
everyone”
• Even though its main work is research in
academia, the door is also wide open for use by
industry
• Predicting traffic congestion or share prices,
simulating human organs, or forecasting the
weather
“The excellent management that SUSE Linux Enterprise Server
provides is one of the key factors behind Tsubame 2.5’s success.”– Professor Satoshi Matsuoka, Global Scientific Information and
Computing Center, Tokyo Institute of Technology
© Copyright 2016 SUSE LLC. All rights reserved.
HPC Impacts Our Lives Through Medical Research
24
PSC’s game-changing supercomputer is being used to
advance medical research
• Visualize data in a way that transforms understanding of
public health issues
• Calculate the electronic structure of a large organic
semiconductor molecule with applications in solar power cells
• Model the possible benefits of flu vaccine choice
• Genetic sequencing of various species
• Assemble 378 billion base pairs of bacterial DNA from the
intestines of healthy patients and those with diabetes
“SUSE Linux Enterprise Server is the only distribution that supports the
full capabilities of the SGI machine. It was a no-brainer for this
application. We use it. We recommend it. SUSE has a newer kernel
than other options, making it the best choice.”– Jim Kasdorf, Director of Special Projects at PSC
© Copyright 2016 SUSE LLC. All rights reserved.
Banking & Financial Services Rely on High Performance Computing
25
The ways in which research applications and financial
services applications rely on Linux and HPC are in
many cases quite similar
• Risk modeling in determining aggregate risk in financial
portfolios
• Fraud detection in real time as millions of transactions are
processed between disparate systems
o Find suspicious patterns in related data sets
o Create and deploy new fraud models into event flows quickly and with
minimal effort
• High-frequency trading applications execute trades
milliseconds before competitors – which can mean millions
in additional returns
• Pricing and regulatory compliance applications
• Big data applications like customer profiling and predictive
analytics
© Copyright 2016 SUSE LLC. All rights reserved.
Beijing Rural
Commercial
Bank
Banking & Financial Services firms running
SUSE Linux Enterprise Server
The SUSE HPC Stack
27
Hardware
SUSE Linux Enterprise Server
© Copyright 2016 SUSE LLC. All rights reserved.
Storage & File Systems Networking Message Passing Interface (MPI)
EXT3 BTRFS
XFS
OCFS2 pNFS
NFS
Ceph FS
pNFS
IBRIX
Lustre
GPFS
10G OFED
TCP Offload
ParTec
HPE
SGI
Intel
MPICH
openMPI
SUSE
supported
SUSE
partner
40G
Queuing / Management Tools
Applications
PBS Pro IBM Spectrum LSF
Moab Bright CM
Grid Engine
SUSE HPC Features, 1 of 2
• Asynchronous I/O (AIO)Form of I/O processing that permits other
processing to continue before the transmission
has finished
• Modular I/O SchedulerAlgorithm most suitable for workload can be
chosen dynamically
• Multi-core/hyper-threading processor
supportExecute threads in parallel within each individual
processor
• Intel I/O AccelerationOffloads the CPU towards the network card, thus
allowing the system to continue processing data
while I/O is taking place
Advanced I/O Processing
• CFQComplete Fair Queuing, suitable for a wide variety
of applications, default I/O scheduler
• Deadline I/O schedulerA valuable choice on a server with intelligent
hardware, or if latency is the primary goal
• Anticipatory I/O scheduler Suitable for file servers but does not work as well
for database workloads
• No-opBetter suited for systems with storage solutions
that have their own cache and their own
virtualization Modular I/O Schedule
Flexible/Pluggable I/O Scheduler
28© Copyright 2016 SUSE LLC. All rights reserved.
SUSE HPC Features, 2 of 2
• Lustre-enabled kernelParallel distributed file system for large-scale
cluster computing & optimal performance
• CPU and system activity managementCPUset System, CPUset command line tool,
Sysstat package, IRQbalance
• Diagnose complex applicationsSystemTap for diagnosing apps, LTTng 2.0, an
open source tracing framework to help gain
insights into system behavior
Performance and Management
• OpenFabrics Enterprise Distribution
(OFED)Remote Direct Memory Access (RDMA) switched
fabric technologies – high-speed data transport
technologies for server and storage connectivity
High-speed Data Transport
29
• Computer memory design used in multiprocessors,
where the memory access time depends on the
memory location relative to a processor
• NUMA provides excellent scalability and performance
for Intel Itanium and AMD Opteron architectures
• Auto-tuning available when apps aren’t NUMA-aware
Non-Uniform Memory Architecture
(NUMA)
© Copyright 2016 SUSE LLC. All rights reserved.
SUSE Linux Enterprise Server for High Performance Computing
Harness the power of supercomputing and solve your most
demanding computational and data-intensive problems.
www.suse.com/products/server/hpc.html
• Accelerate innovation with a strong ecosystem of partners.
• Optimize scalability and performance with Linux clustering.
• Realize faster time to value through simplified configuration of HPC workloads.
50% Reduction in administration
20%Boost in productivity
99.999%Stable and reliable
© Copyright 2016 SUSE LLC. All rights reserved.
30
Outlook
• OpenHPC community contributions will
continue to grow
• Foster growing partner ecosystem
• Wider HPC adoption across many industries
• Increase in hybrid cloud deployments mixed
with on-premise
• More definition around HPC Module packaging
© Copyright 2016 SUSE LLC. All rights reserved.
32
High Performance Computing Ecosystem
Partnering
OpenHPC
community
ARM
Microsoft
Intel’s
enterprise HPC
based on
OpenHPC
community &
SLES
Modularize
HPC to focus
on specific
needs
HPC partners
leverage SLES
SLES is the
foundation for
HPC
© Copyright 2016 SUSE LLC. All rights reserved.
33
34
Learn more at www.suse.com/products/server/hpc.html
34
Questions?
© Copyright 2016 SUSE LLC. All rights reserved.
ResourcesSUSE Linux Enterprise Server for High Performance Computing
Products and Services for HPC• https://www.suse.com/products/server/hpc.html
• https://wiki.microfocus.net/index.php?title=SUSE_High_Perf
ormance_Computing
Campaigns and Programs for HPC
• https://www.suse.com/promo/sle/high-performance-
computing
HPC Success Stories
• https://www.suse.com/success/#highperformancecomputing
35
SUSE Webinar
• https://www.brighttalk.com/webcast/11477/
223349
HPC Partners
• http://openhpc.community/
• https://www.suse.com/promo/intel-suse-
hpc-stack/
• https://azure.microsoft.com/en-
us/marketplace/partners/suse/suselinuxent
erpriseserver12optimizedforhighperforman
cecompute/
37
Unpublished Work of SUSE LLC. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE LLC.
Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their
assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated,
abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE.
Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.
General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a
product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making
purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and
specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The
development, release, and timing of features or functionality described for SUSE products remains at the sole discretion
of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time,
without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this
presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-
party trademarks are the property of their respective owners.