Renaissance of sparc UKOUG 2014

transcript

The Renaissance of

SPARC Björn Rost / Philippe Fierens

Stories from the field

Not so safe harbour ;-)

About Björn

• Founder, manager and consultant

• President RAC SIG

• ACE Director

About Phil

Philippe Fierens Independent

Oracle DBA (not solaris admin) Architect

and Oracle ACE Associate

and most of all DAD & husband ;-)

AND …

whoami

AgendaHistoric overview

SPARC features

Comparing performance : stories from the field

Licensing in a multi core multi socket world

Sparc Super Cluster Exadata ++

Solaris : well ahead of comet

The near (R)evolution

How it all began

ssor A

RChitec

RC V7 s

RC V9 a

rchite

SPARC In

l http

Oracle

2004 2005

Dual c

ltra I

ries l

h Ultr

2010 2011 2012 2013

M7M7#Processor# •  32#SPARC#S4#Cores#

–  #Dynamic#Threading#1#to#8#threads#/#core#

•  New#Cache#Organiza8ons#–  Shared#L2#Data#and#Instr#Cache#–  64#MB#Shared#and#Par@@oned#L3#Cache#

•  DDR4#DRAM#–  Upto#2TB#Physical#Mem#/#Processor#–  2x#F3x#beHer#Mem#Bandwidth#over#Last#gen#

•  Applica8on#Accelera8on#–  Real@me#Applica@on#Data#Integrity#–  Concurrent#Memory#Migra@on##–  VA#masking#–  DB#Query#Offload#Engine#

•  SMP#1#to#32#Processors#•  Technology#20nm#

more about this later

SPARC FEATURES

Reliability

Availability

Serviceability

RAS• Redundant hardware in combination with software (FMA)

Fault Management Architecturepredictive self-healing, offline resources

with increased correctable errors

• Reducing components

• Reduce connectors to ASICS boards

• Instruction level retry and ECC• Hot-swapping / plug-ability

• PCI cards• hard disks• power supplies

SPARC Server Availability M6-32 / M5-32: Highest Availability in the SPARC Server Portfolio

T5#8$T5#4$T5#2$

T4#2$T4#1$ M6#32$$

M5#32$

VM Secure Live Migration Redundant / Hot-swap Fans, Power, Disks

ECC, Predictive Self Healing

Physical Partitions / Redundant Clock / Redundant & Hot-Plug SP Hot Plug PCIe Lane Sparing (L3$/Memory/Interconnect) + Redundant Interconnect

64 bit support (1994)•64 bit support in HW and Solaris

1991 1992 1994 1995 2001

large memory support

•64 bit allows large address space•Large pages support by default•SPARC 2GB large pages

% pagesize -a8192 <-- 8K65536 <-- 64K4194304 <-- 4M268435456 <-- 256M2147483648 <-- 2G

memory per CPU

x4170 M2 (nehalem)

X3-2 (sandy bridge)

X4-2 (sandy bridge)

3-channel Intel E5

T4 (S3)

T5 (S3)

M6 (S3)

M7 (S4)

0 512 1024 1536 2048

2048GB

1024GB

memory per CPU

Solaris 10 (2005)

• Supports SPARC and x86

• Scalable kernel

• ZFS

• Dtrace

• Zones lightweight virtualization

• >12 years of premier support

• +3 years of extended support

OVM for SPARC (2007)

• Logical Domains

• Hypervisor built in hardware

• Zero overhead

• Strict isolation of CPU, mem, IO

• Flexible vswitch “stuff”

• PCIe dio for HBAs

• Improved vdisk performance (sol11 SRU19)

• Supported for hard partitioning

OVM types/roles• Control domain :

primary domain used to connect to HV and manage resources

• I/O domain :Direct access to phys I/O devices

PCI root complex PCI card Regular non virtualized driversLimited number of I/O domains depending on platform T5-2

vs T5-4• Service domain :

Provides virtual I/O to guestsVirtual networkVirtual Disk

• Guest domain => consumer of virtual I/O 20

SPARC capping

SPARC offers lots of options to CAP CPU

❖ PDOMS

❖ LDOMS

❖ Zones

• Full hardware and software fault isolation• Divide large server into smaller one• Physical segregation

• CPU• memory• I/O resources

ZonesGlobal zone

“owns” the hardware and controls the resources

Non Global zonesConfigured and run in global zone Exclusive access and Shared network access possible, NO Exclusive access to HBA Share same kernel as global zone Isolated ( != kernel zones)

Supported to CAP CPU (licensing)

domain isolation

M9000 with 4 domains

PRODRAC

STAGINGRAC

ACCEPTANCERAC

PRODRAC

STAGINGRAC

ACCEPTANCERAC

ZONES ZONES

subserver licensing

68.000

1.221.760

SW license EE,RAC,PART,DIAG, TUNE + 3years supportHW 2*T4-1, 128GB RAM, 3 years support

❖ Rule: Oracle SW costs will always outweigh GW

❖ Standard Edition: get as many cores per CPU as possible

❖ Enterprise Edition: get the smallest number of cores with most power per core

dark ages of SPARC

• Single thread Ultra systems EOL after 05

• T1/T2 not really general purpose

• Slow single thread performance

• Mx000 the first successor

CMT explained

psrinfo0 on-line since 12/07/2014 23:34:441 on-line since 12/07/2014 23:34:46

snipped …

60 on-line since 12/07/2014 23:34:4661 on-line since 12/07/2014 23:34:4662 on-line since 12/07/2014 23:34:4663 on-line since 12/07/2014 23:34:46

perf benchmarkstim

threads per core

0,5 1 2 4

x5570 @2.93GHzT4 @2.85GHz29

results - latency

• Very simple test w looped SQL

• Intel wins at very low utilization

• 90s vs 119s (30%)

• this is intel turbo-boost effect

• @ 1 thread/core same response time

• SPARC performs better at over-saturation

more benchmarks

http://kevinclosson.wordpress.com/2012/02/06/introducing-slob-the-silly-little-oracle-benchmark/31

• SLOB

• LIO test mode

• there are “modes” for PIO, writes, ...

• One suite of tools for multiple platforms

• Very simple (and open) logic

• Easy to set up, easy to compare

SLOB resultspeak SLOBops/s per core

x5570 E5-2640 E5-2690 SPARC64 T4 T5

SLOB resultstime to complete 500k SLOB loops

threads per core

0,25 0,5 1 2 4 8

E5-2640 T4 T533

Solaris 11 (2010)

❖ Network stack re-written❖ ipadm/dladm❖ virtual networking

❖ Enhanced support for zones❖ kernel zones❖ immutable zones …❖ Vnics

❖ New IPS packaging system❖ GNU tools by default (feels like linux)❖ Puppet integration❖ Openstack integration❖ ZFS enhancements

❖ deduplication

golden ages of sparc

• T4

• T5

• M7

2010 2011 2012 2013

M7M7#Processor# •  32#SPARC#S4#Cores#

–  #Dynamic#Threading#1#to#8#threads#/#core#

•  New#Cache#Organiza8ons#–  Shared#L2#Data#and#Instr#Cache#–  64#MB#Shared#and#Par@@oned#L3#Cache#

•  DDR4#DRAM#–  Upto#2TB#Physical#Mem#/#Processor#–  2x#F3x#beHer#Mem#Bandwidth#over#Last#gen#

•  Applica8on#Accelera8on#–  Real@me#Applica@on#Data#Integrity#–  Concurrent#Memory#Migra@on##–  VA#masking#–  DB#Query#Offload#Engine#

•  SMP#1#to#32#Processors#•  Technology#20nm#

M7 in a nutshell

Applica'on Data Integrity Virtual Address (VA) Masking Database Query Accelerator Engine

In memory accelera'on Value Range Comparisons

Fine Grain Memory Migra'on On-‐Chip Decompression

2TB memory per chip32 cores on 1 chip

SOFTWARE IN SILICON

Test drive the M7

https://swisdev.oracle.com

SSC line-up

M6-32T5-8T4-440

SSC aka Exadata ++

Same exadata storage cells as in Exadata

Flexibility

2 Types :

A Technical Overview of the Oracle SPARC SuperCluster T4-4

Application Domain configuration.

Figure 9. Domains mapped to a half rack system.

Notice how the CPU and memory is configured separately from the Root Complex. When SPARC SuperCluster is installed, allocation granularity is 25% (8 CPU cores and 256GB of memory) per domain. This means the number and types of domains, number of Root Complexes, cores and memory must be decided upon at installation time. The customer can reassign CPU and memory after the installation using a standalone tool, setcoremem. The granularity is 4 CPU cores and 32GB of memory.

Configurations

SPARC SuperCluster is a complete solution that has been designed, tested and integrated to run the Oracle software stack, third party ISV software, as well as any customer developed software. Oracle has tested, tuned, and optimized the best possible hardware and software combinations so customers can take advantage of this integration resulting in a shorter deployment time and better utilization.

Figure 10 shows the components of SPARC SuperCluster T4-4. The numbered items in the figure are identified in the table that immediately follows the figure.

DB Domains -—> IB connect exadata storage cells

App Domains —-> classical SAN based storage

Linux on Sparc?"We think Sparc will become clearly the best chip for running Oracle software. At that point we'd be nuts not to move Oracle Enterprise Linux there. We're a ways away, but I think that's definitely going to happen," Larry Ellison in 2010

http://www.pcworld.com/article/212564/article.html 42

Linux on Sparc?

42:linux-uek3-3.8 brost$ git log -SAthena commit aad464a5ef7e5ea745b816852029d830073c6d94 Author: Allen Pais <allen.pais@oracle.com> Date: Wed Mar 6 15:17:59 2013 +0530

correctly recognize SPARC64-X chips The following patch adds support for correctly recognizing SPARC-X chips. cpu : Unknown SUN4V CPU fpu : Unknown SUN4V FPU pmu : Unknown SUN4V PMU Signed-off-by: Katayama Yoshihiro <kata1@jp.fujitsu.com> Signed-off-by: Allen Pais <allen.pais@oracle.com>

42:linux-uek3-3.8 brost$ git log -SSPARC commit a330ea9963a7bce076b92372b35606223997a5b1 Author: Martin K. Petersen <martin.petersen@oracle.com> Date: Wed Feb 26 14:28:25 2014 -0500

ixgbe: Look up MAC address on SPARC systems On SPARC systems first attempt to look up the MAC address in OpenPROM. If that fails resort to using the IDPROM value. Orabug: 18182472 Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Acked-by: Jerry Snitselaar <jerry.snitselaar@oracle.com> Signed-off-by: Ethan Zhao <ethan.zhao@oracle.com>

SPARC not just SUN/Oracle

❖ Fujitsu M10 “Athena”

❖ OpenSparc

❖ study CMT design on FPGA boards!

❖ embedded systems

❖ automotive/industrial

❖ LEON for Space Agencies

SPARC IS

Renaissance of sparc UKOUG 2014

Software