36
HPC Trends for 2010: On the Rebound Actionable Market Intelligence for High Productivity Computing Addison Snell, [email protected] March 2010
HPC Trends for 2010:On the Rebound
HPC Trends for 2010:On the Rebound
Actionable Market Intelligence for High Productivity Computing
Actionable Market Intelligence for High Productivity Computing
Addison Snell, [email protected]
March 2010
“InterSect360”?“InterSect360”?
• Chris Willard and I acquired the Tabor Research division of Tabor Communications (HPCwire) in July
• Changed the company name, not the business• Enduring partnership with Tabor Communications
– Exclusive market research partnership drives user-based research methodology
– Venue for feature articles and blogs– Weekly podcast: “HPCwire Soundbite”
• Still able to (and anxious to) work with other excellent pubs, like insideHPC
• Chris Willard and I acquired the Tabor Research division of Tabor Communications (HPCwire) in July
• Changed the company name, not the business• Enduring partnership with Tabor Communications
– Exclusive market research partnership drives user-based research methodology
– Venue for feature articles and blogs– Weekly podcast: “HPCwire Soundbite”
• Still able to (and anxious to) work with other excellent pubs, like insideHPC
TopicsTopics
• Top-level HPC market dynamics• Technology areas to watch in 2010• Entry-level and midrange HPC expansion (the
“missing middle”)• Where to look for growth
• Top-level HPC market dynamics• Technology areas to watch in 2010• Entry-level and midrange HPC expansion (the
“missing middle”)• Where to look for growth
2009 HPC Market Model2009 HPC Market Model
• 2009 sucked: Down 20% from $19.0B to $15.2B• (Still finalizing model and forecast)• BUT:
– Most of decline was due to lengthening sales cycles
– Short-term market effects have to do with how fast people buy, not how much
– Areas of relative strength (less loss):• Government• Supercomputers• Storage
• 2009 sucked: Down 20% from $19.0B to $15.2B• (Still finalizing model and forecast)• BUT:
– Most of decline was due to lengthening sales cycles
– Short-term market effects have to do with how fast people buy, not how much
– Areas of relative strength (less loss):• Government• Supercomputers• Storage
Supercomputing: Are We There Yet?Supercomputing: Are We There Yet?
• I agree with Mike Bernhardt– We’re totally there. Look what we can do!– We’ll never be there. Look what we still need to
do.
• The difference between HPC and enterprise:– HPC: Once you’ve designed the bridge, you
don’t need to design it again– Enterprise: Once it works the first time, for
heaven’s sake, don’t touch it!
• This is why HPC budgets have been mostly stable (albeit slow to approve)
• I agree with Mike Bernhardt– We’re totally there. Look what we can do!– We’ll never be there. Look what we still need to
do.
• The difference between HPC and enterprise:– HPC: Once you’ve designed the bridge, you
don’t need to design it again– Enterprise: Once it works the first time, for
heaven’s sake, don’t touch it!
• This is why HPC budgets have been mostly stable (albeit slow to approve)
Important (But Not Necessarily Disruptive)
Important (But Not Necessarily Disruptive)
• Technologies we’re watching in HPC in 2010:– Workstations– SMPs, virtual and physical– Windows– Cloud / SaaS / Utility– Infiniband– Accelerators– File systems
• Technologies we’re watching in HPC in 2010:– Workstations– SMPs, virtual and physical– Windows– Cloud / SaaS / Utility– Infiniband– Accelerators– File systems
WorkstationsWorkstations
• Remember when workstations were used for work?
• In the salad days, beefy technical workstations cost $10K - $30K and engineers did a lot of the design workflow on them
• Systems like Cray CX-1, SGI Octane 3, NVIDIA Tesla have the ability to fill this product gap again
• Addresses a critical adoption problem for ELMR: Building a bridge from CAD to CAE.
• Remember when workstations were used for work?
• In the salad days, beefy technical workstations cost $10K - $30K and engineers did a lot of the design workflow on them
• Systems like Cray CX-1, SGI Octane 3, NVIDIA Tesla have the ability to fill this product gap again
• Addresses a critical adoption problem for ELMR: Building a bridge from CAD to CAE.
SMPsSMPs
• We are seeing renewed interest in SMPs on the user side, along with products on the vendors side
• True SMPs from [deleted] and SGI• Virtual SMPs: ScaleMP, 3leaf, Symmetric
Computing
• “We’re going up in cores and memory but down in node count.” – Pharmaceutical HPC user
• SMPs can also help with new HPC adoption
• We are seeing renewed interest in SMPs on the user side, along with products on the vendors side
• True SMPs from [deleted] and SGI• Virtual SMPs: ScaleMP, 3leaf, Symmetric
Computing
• “We’re going up in cores and memory but down in node count.” – Pharmaceutical HPC user
• SMPs can also help with new HPC adoption
Architectures - Percent of Usage Org1 Org2 Org3 Org4 Org5 Org6 Org7 AvgClusters or blades 98% 10% 100% 85% 15% 20% 80% 58%SMP systems (includes vector-based systems) 1% 0% 0% 10% 85% 75% 20% 27%MPP 0% 0% 0% 0% 0% 0% 0% 0%Grid/Cloud (systems not dedicated to HPC) 1% 90% 0% 5% 0% 5% 0% 14%Other: please specify 0% 0% 0% 0% 0% 0% 0% 0%
TOTAL 100% 100% 100% 100% 100% 100% 100% 100%
HPC Norms and Best Practices in Pharmaceuticals,InterSect360 Research, 2010
WindowsWindows
• We still say this is coming• Early adoption has begun in entry-level
commercial users and certain vertical markets (e.g. finance)
• Linux continues to fragment• Transition from Linux to Windows is harder
than Unix to Linux; it takes a while• Is it possible that Windows could outperform
Linux at scale in two to three years?
• We still say this is coming• Early adoption has begun in entry-level
commercial users and certain vertical markets (e.g. finance)
• Linux continues to fragment• Transition from Linux to Windows is harder
than Unix to Linux; it takes a while• Is it possible that Windows could outperform
Linux at scale in two to three years?
CloudCloud
• First of all, take it easy. This is an evolution from grid, utility, etc., and utility models aren’t new
• Cloud is not an application or a market in itself, but rather an access or usage model
• Cloud: Outsourcing part of your IT infrastructure or workflow through a web (or web-like) interface.
• Top system purchase criteria: performance, reliability
• Vendors: What does cloud mean to your products?• Users: What are you trying to achieve? Is cloud
the right way to do it?
• First of all, take it easy. This is an evolution from grid, utility, etc., and utility models aren’t new
• Cloud is not an application or a market in itself, but rather an access or usage model
• Cloud: Outsourcing part of your IT infrastructure or workflow through a web (or web-like) interface.
• Top system purchase criteria: performance, reliability
• Vendors: What does cloud mean to your products?• Users: What are you trying to achieve? Is cloud
the right way to do it?
In Defense of ISVsIn Defense of ISVs
• Who here thinks they should pay more for software?
• But consider, industry trends have made things tougher for ISVs:– SMPs to clusters– Unix to Linux to Windows– Multi-core; how are the cores being used?– Accelerators
• Nevertheless, we see ISVs responding– New licensing models– SaaS
• Who here thinks they should pay more for software?
• But consider, industry trends have made things tougher for ISVs:– SMPs to clusters– Unix to Linux to Windows– Multi-core; how are the cores being used?– Accelerators
• Nevertheless, we see ISVs responding– New licensing models– SaaS
Study of SaaS Outlook Among HPC ISVs
Study of SaaS Outlook Among HPC ISVs
• ISVs raise your hands. I’d like to talk to you.• These are fundamental questions for cloud in
HPC: – How will I run the application?– What is the licensing model?– Who hosts it?
• High-level findings will be published in debut issue of “HPC in the Cloud,” coming from Tabor Communications next month
• Many ISVs already responding to this (Altair, Exa)
• ISVs raise your hands. I’d like to talk to you.• These are fundamental questions for cloud in
HPC: – How will I run the application?– What is the licensing model?– Who hosts it?
• High-level findings will be published in debut issue of “HPC in the Cloud,” coming from Tabor Communications next month
• Many ISVs already responding to this (Altair, Exa)
Infiniband as a System InterconnectInfiniband as a System InterconnectFigure 4. Percent of Systems by System Interconnect and Year Acquired
N=varies by year – 14,18,32,26,45,27 for years 2003 to 2008 Source: InterSect360 Research, 2009
InterSect360 Research HPC User Site Census: Interconnects
Infiniband as Storage InterconnectInfiniband as Storage InterconnectFigure 7: Maximum Site-Level Storage by Primary Network
N=128 Source: InterSect360 Research, 2010
InterSect360 Research HPC User Site Census: Storage
AcceleratorsAccelerators
• GPUs getting most of the attention, but FPGAs still have compelling value in some areas
• Where they’re good:– GPUs excel at ELMR, floating-point intense– FPGAs excel at high end, text and integer intense
• Addressing the programming hurdle– GPUs: CUDA– FPGAs: embedded in architectures (e.g. Convey,
XtremeData, BlueArc)
• Rising tide floats both boats (for now)
• GPUs getting most of the attention, but FPGAs still have compelling value in some areas
• Where they’re good:– GPUs excel at ELMR, floating-point intense– FPGAs excel at high end, text and integer intense
• Addressing the programming hurdle– GPUs: CUDA– FPGAs: embedded in architectures (e.g. Convey,
XtremeData, BlueArc)
• Rising tide floats both boats (for now)
Challenges for AcceleratorsChallenges for Accelerators
• Overcoming latency hit• Programmability (still)• Maintaining pace of development• I had been anticipating an entertaining war
between Intel Larrabee and NVIDIA, but …
• Overcoming latency hit• Programmability (still)• Maintaining pace of development• I had been anticipating an entertaining war
between Intel Larrabee and NVIDIA, but …
File Systems in HPCFile Systems in HPC
• This has been a highly fragmented market• An early Tabor Research study found 17
different file systems installed at only 38 sites. (Quick, name 12.)
• The emergence of clustered, parallel file systems provides touchpoints for continued consolidation:– GPFS– Lustre– pNFS
• This has been a highly fragmented market• An early Tabor Research study found 17
different file systems installed at only 38 sites. (Quick, name 12.)
• The emergence of clustered, parallel file systems provides touchpoints for continued consolidation:– GPFS– Lustre– pNFS
Clustered File Systems in HPCClustered File Systems in HPC
• Lustre has most mentions in InterSect360 Research Site Census report– Followed by GPFS– Lustre well-aligned with academic sector
• Share of licensing revenue is probably approximately even between Lustre and GPFS
• Panasas (PanFS) currently in third position for parallel, clustered file systems
• EMC has strong market position, but EMC MPFS is not a true parallel, clustered file system competitor
• Lustre has most mentions in InterSect360 Research Site Census report– Followed by GPFS– Lustre well-aligned with academic sector
• Share of licensing revenue is probably approximately even between Lustre and GPFS
• Panasas (PanFS) currently in third position for parallel, clustered file systems
• EMC has strong market position, but EMC MPFS is not a true parallel, clustered file system competitor
pNFS EmergencepNFS Emergence
• Still very early in pNFS adoption. We have seen it in a select few installations, but really it’s still in beta.
• Some of the previously dominant voices in favor of pNFS have gotten quieter.
• NetApp, a primary proponent of pNFS, has decided not to focus on the HPC market
• Panasas and BlueArc are positioned to carry it forward
• Still very early in pNFS adoption. We have seen it in a select few installations, but really it’s still in beta.
• Some of the previously dominant voices in favor of pNFS have gotten quieter.
• NetApp, a primary proponent of pNFS, has decided not to focus on the HPC market
• Panasas and BlueArc are positioned to carry it forward
LustreLustre
• Strong presence in market, but now defocused:– Sun bought CFS and “owned” Lustre, but Lustre
talent left and Sun focused storage strategy on QFS
– Oracle seems to take Sun even further from Lustre, and away from HPC
• There will continue to be open source groups and development, but without much corporate focus
• Cray is aligned with Lustre, and ORNL is a center of excellence
• Strong presence in market, but now defocused:– Sun bought CFS and “owned” Lustre, but Lustre
talent left and Sun focused storage strategy on QFS
– Oracle seems to take Sun even further from Lustre, and away from HPC
• There will continue to be open source groups and development, but without much corporate focus
• Cray is aligned with Lustre, and ORNL is a center of excellence
HP / IBRIXHP / IBRIX
• IBRIX had little to no presence in HPC prior to acquisition.
• IBRIX, like Polyserve, gives HP datacenter options that are relevant from a facilities standpoint.
• IBRIX had little to no presence in HPC prior to acquisition.
• IBRIX, like Polyserve, gives HP datacenter options that are relevant from a facilities standpoint.
GPFSGPFS
• GPFS gives IBM HPC a credible, meaningful competitive advantage in many situations
• IBM has more advantage before pNFS is established
• IBM could make a bold move with GPFS immediately, while they have file system leadership
• But to what extent can they push GPFS independently? Most users want their storage software to come with their storage hardware.
• Partnership with DataDirect increases presence.
• GPFS gives IBM HPC a credible, meaningful competitive advantage in many situations
• IBM has more advantage before pNFS is established
• IBM could make a bold move with GPFS immediately, while they have file system leadership
• But to what extent can they push GPFS independently? Most users want their storage software to come with their storage hardware.
• Partnership with DataDirect increases presence.
One More Thought on File Systems: Tape
One More Thought on File Systems: Tape
• Tape: The Storage You Don’t Think About• “WORN”: Write Once, Read Never• Data is getting not lost, only forgotten• Ability to put a file system on tape creates a
potential market for “active archives”• Dr. Goh mention active archives at NASA at
9:48 a.m. today!
• Tape: The Storage You Don’t Think About• “WORN”: Write Once, Read Never• Data is getting not lost, only forgotten• Ability to put a file system on tape creates a
potential market for “active archives”• Dr. Goh mention active archives at NASA at
9:48 a.m. today!
The “Missing Middle”The “Missing Middle”
• We have extensively studied the entry-level and midrange (ELMR) segments
• Not as big as first appears:– 50% of revenue, but:– About two-thirds of ELMR
shipments are upgrades or additions1
– 20-25% of ELMR systems are bought by users that also have larger systems2
• We have extensively studied the entry-level and midrange (ELMR) segments
• Not as big as first appears:– 50% of revenue, but:– About two-thirds of ELMR
shipments are upgrades or additions1
– 20-25% of ELMR systems are bought by users that also have larger systems2
Revenue share by product classInterSect360 Research, 2010
Only 10% to 15% of system
revenue goes to true ELMR users
Only 10% to 15% of system
revenue goes to true ELMR users
1 InterSect360 Research HPC User Site Census: Lifecycles2 Custom study: Entry-level and Midrange, 2008
2009Supercomputers 20%High-End 30%Midrange 27%Entry Level 23%
Midrange, Entry, and Sub-EntryMidrange, Entry, and Sub-Entry
Do you consider yourself to be an HPC user?
Yes61%
No39%
n = 95
Some Sub-entry users consider themselves HPC, whereas some Midrange users do not.
Some Sub-entry users consider themselves HPC, whereas some Midrange users do not.
Highest level of system in use
16%
38%
31%
15%
Sub-entry (PCs, workstations)
Entry HPC (under 40 proc., $50K)
Midrange HPC (40 - 200 proc., $50K - $250K)
High-end HPC (over 200 proc., $250K)
n = 95
InterSect360 Research (as Tabor Research), 2008
Size of CompanySize of Company
Total Govt Acad Comm Mid Entry Sub
Under 100 4% 0% 7% 4% 3% 0% 17%
100 to 999 36% 20% 48% 36% 26% 41% 50%
1,000 to 9,999 36% 50% 26% 36% 41% 38% 17%
10,000 to 99,999 20% 25% 15% 21% 26% 14% 17%
Over 100,000 4% 5% 4% 4% 3% 7% 0%
Number of respondents: 75 20 27 28 34 29 12
Table 4: Size of Organization, by Number of EmployeesSector Level of HPC
Source: Tabor Research, February 2008
About one-quarter of respondents come from large organizations; about half of these also
purchase high-end HPC systems.
About one-quarter of respondents come from large organizations; about half of these also
purchase high-end HPC systems.
Not “HPC,” but “R&D”Not “HPC,” but “R&D”
Total Govt Acad Comm Mid Entry Sub
5: Strongly agree 43% 33% 36% 55% 47% 39% 42%
4: Agree 32% 42% 27% 29% 35% 30% 26%
3: Neutral 17% 4% 27% 16% 9% 27% 16%
2: Disagree 3% 8% 3% 0% 5% 3% 0%
1: Strongly disagree 5% 13% 6% 0% 5% 0% 16%
Number of respondents: 95 24 33 38 43 33 19
Mean: 4.0 3.8 3.8 4.4 4.1 4.1 3.8
Source: Tabor Research, February 2008
Table 10: Agree/Disagree with the Following Statement:"Leadership in R&D or engineering is critical to the success of our organization."
Level of HPCSector
1. “R&D leadership” is a better overall message than “HPC.”
2. Especially among commercial users.3. And it correlates with actual HPC adoption.4. But it’s not everybody. And that’s good.
1. “R&D leadership” is a better overall message than “HPC.”
2. Especially among commercial users.3. And it correlates with actual HPC adoption.4. But it’s not everybody. And that’s good.
Satisfaction Gaps: Purchase CriteriaSatisfaction Gaps: Purchase Criteria
Total Govt Acad Comm Mid Entry Sub
Number of processors 6 8 13 -2 -6 3 41
Total memory -23 -21 -23 -24 -39 -25 18
Industry-standard benchmarks 3 -13 36 -12 -10 -6 53
Benchmarked application performance 10 8 32 -8 -14 6 76
Total cost of ownership (TCO) 4 21 -10 5 -5 -6 47
Facilities issues (floor space, power consumption, cooling, noise) 9 0 35 -8 -11 19 41
Services 1 -17 0 14 -14 1 42
Support -25 -13 -21 -38 -34 -9 -35
Delivery time 15 -4 22 22 -8 27 47
Evaluation of vendor 0 8 -5 0 2 -3 0
Number of respondents 92 24 31 37 42 33 17
Overall satisfaction by segment: 0 -2 8 -5 -14 1 33
Satisfaction Gap = (Satisfaction Score - Importance Score) x 100Table 49: Satisfaction Gaps: Purchase Criteria
Sector Level of HPC
Source: Tabor Research, February 2008
Satisfaction Gaps: TCO MetricsSatisfaction Gaps: TCO Metrics
Total Govt Acad Comm Mid Entry Sub
Server (hardware) price -13 -25 -19 0 -16 -27 24
System software price -7 17 -35 0 -7 -18 12
Third-party application software licensing -22 4 -49 -17 -27 -15 -24
Ability to upgrade -5 -17 10 -8 -21 6 17
Total uptime -19 -42 -19 -5 -17 -31 -6
Warranty and support price -9 4 -26 -2 -19 -6 12
Power consumption 11 13 3 16 -14 25 47
Cooling -12 -25 -29 11 -14 -30 29
Noise level or heat in working environment 20 17 16 24 17 6 53
Rack space 22 17 0 43 14 19 47
Personnel: system administrators 4 21 -9 5 0 -3 29
Personnel: application programmers 8 17 -29 35 9 -9 41
Number of respondents 92 24 31 37 42 33 17
Overall satisfaction by segment: -2 0 -16 9 -8 -7 24
Satisfaction Gap = (Satisfaction Score - Importance Score) x 100Table 40: Satisfaction Gaps: TCO Metrics
Level of HPCSector
Source: Tabor Research, February 2008
Not Just the Hardware and SoftwareNot Just the Hardware and Software
• Imagine a sub-entry user with a clear case for HPC, and you gave them FREE hardware and software– What data will they use?– How will they test their models?– Do they have the organizational will to change
their product design workflow?
• It takes years to successfully integrate HPC into a workflow, and it tends to grow organically within companies
• Imagine a sub-entry user with a clear case for HPC, and you gave them FREE hardware and software– What data will they use?– How will they test their models?– Do they have the organizational will to change
their product design workflow?
• It takes years to successfully integrate HPC into a workflow, and it tends to grow organically within companies
Examples from a Mature Vertical: Pharma
Examples from a Mature Vertical: Pharma
• “We have four different research centers – different therapeutic areas – each with its own HPC. There is collaboration at the research level – we share best practices – but the execution is independent. The different sites have different histories and cultural expertise.”
• “We have many sites using their own clusters. The problems start when licenses are shared for everyone, like when a user in one location cannot run his job because the licenses are being used by someone in another location who started his job days ago.”
• “There are dedicated HPC clusters for some groups, dedicated to doing analysis from sequencers or other instruments. If a groups has a specialized 24/7 need for HPC, they build their own clusters. There aren’t a lot of these, maybe half a dozen. One reason is problems with large data transfer. Some groups want to sit next to their own data.”
• “We have four different research centers – different therapeutic areas – each with its own HPC. There is collaboration at the research level – we share best practices – but the execution is independent. The different sites have different histories and cultural expertise.”
• “We have many sites using their own clusters. The problems start when licenses are shared for everyone, like when a user in one location cannot run his job because the licenses are being used by someone in another location who started his job days ago.”
• “There are dedicated HPC clusters for some groups, dedicated to doing analysis from sequencers or other instruments. If a groups has a specialized 24/7 need for HPC, they build their own clusters. There aren’t a lot of these, maybe half a dozen. One reason is problems with large data transfer. Some groups want to sit next to their own data.”“HPC Norms and Best Practices in the Pharmaceutical Industry,” InterSect360 Research, 2010
If I Were the KingIf I Were the King
• I’d build a program to link university engineering courses with local companies:– Creation of that scalable digital models of
companies’ products is part of the coursework– Students must test the models; corroborate data– Start on multi-core PCs; scalable resources via
“grid”– Coursework includes a subsidized internship– Companies get to use the results– Creates a talent pool. Give a bonus to students who
go to work for those companies full time after school.
• I’d build a program to link university engineering courses with local companies:– Creation of that scalable digital models of
companies’ products is part of the coursework– Students must test the models; corroborate data– Start on multi-core PCs; scalable resources via
“grid”– Coursework includes a subsidized internship– Companies get to use the results– Creates a talent pool. Give a bonus to students who
go to work for those companies full time after school.
Where to Look for GrowthWhere to Look for Growth
• Much of the “Missing Middle” discussion centers on engineering and supply chains. Consider inclusive looks at “engineering”:– Consumer product manufacturing. (Where is
R&D leadership critical?)– Chemical engineering– Bio-engineering– Agri-engineering– Engineering services
• Much of the “Missing Middle” discussion centers on engineering and supply chains. Consider inclusive looks at “engineering”:– Consumer product manufacturing. (Where is
R&D leadership critical?)– Chemical engineering– Bio-engineering– Agri-engineering– Engineering services
Expanding Our CommunityExpanding Our Community
• InterSect360 Research is forming a Modeling and Simulation Leadership Panel– A worldwide panel of organizations using
computational modeling, simulation, and analytics to advance their leadership positions in engineering development and scientific research.
– Build sense of community– Provides access to analyst research, peer
expertise– Interested? Contact me or Mike Bernhardt
• InterSect360 Research is forming a Modeling and Simulation Leadership Panel– A worldwide panel of organizations using
computational modeling, simulation, and analytics to advance their leadership positions in engineering development and scientific research.
– Build sense of community– Provides access to analyst research, peer
expertise– Interested? Contact me or Mike Bernhardt
Conclusions and Free Marketing Advice
Conclusions and Free Marketing Advice
• HPC, at all levels, is about leadership• Do not think of entry-level users as small: To
them this is the biggest system they’ve ever bough
• A supercomputer without an application is an expensive space heater
• An application without data is only a nice theory
• How do your users (not only supercomputing level) exemplify leadership?
• HPC, at all levels, is about leadership• Do not think of entry-level users as small: To
them this is the biggest system they’ve ever bough
• A supercomputer without an application is an expensive space heater
• An application without data is only a nice theory
• How do your users (not only supercomputing level) exemplify leadership?
Actionable Market Intelligence for High Productivity Computing
Actionable Market Intelligence for High Productivity Computing
