Analyzing Commercial Processor Performance Numbers for Predicting Performance of Applications of Interest
Kenneth Hoste*, Lieven Eeckhout* and Hendrik Blockeel°* Ghent University, Belgium - ° K.U. Leuven, Belgium
benchmark
suite
normalized
µarch-indep.
chars
application
of interest
normalized
µarch-indep. chars
genetic
algorithm
performance
numbers
benchmark spacew
weights for characteristics
Performance Estimation for Applications of Interest
Analyzing Commercial Processor Performance Numbers
Case study: Intel desktop processors
Current Practice in Benchmarking
Performance numbers are projected into a three-dimensional space using principal components analysis (PCA). Each point represents one of the 1181 machines, on which the SPEC CPU2000 benchmark suite was run (data obtained from www.spec.org). Choosing colors according to processor characteristics yields valuable insights.
Principal Components Analysis (PCA)Principal components analysis is a statistical technique which computes new dimensions (PCs) with the following properties:
Var[PC1] ≥ Var[PC2] ≥ ... Var[PCn]Cov[PCi,PCj] = 0 (i ≠ j)
Each principal component (PC) is a weighted linear sum of input dimensions. Retaining a limited number of PCs allows reduce the dimensionality of data while controlling the amount of information lost.We retain 3 PCs, which explain 92.5% of total variance.
Cluster analysis Using cluster analysis (k-means) in the three-dimensional space described above, various groups of similar performing machines are identified.
For each cluster of similar machines, a performance model is constructed. Simply grouping machines by ISA or average performance is insufficient, because of the large variety among machine models (for example, Intel Pentium 4 and Intel Xeon).
1st Principal Component
2nd
Prin
cipa
l Com
pone
nt
3rd Principal Component
1st Principal Component
2nd
Prin
cipa
l Com
pone
nt
!!
!
!
!
!
!
!
!!!
! !!!
!
!!!
!!!!!!!
!!!! ! !!!
!
!!
!
! !
!!!!!
!!
!
!
!!!!!
!! !
!! !!
!!! !!
!
!!!
!!
!
!!!!
!!
!!
!
!!
!!
!!
! !
!!
!
!!!
!!!
!!!
!!
!!
!
!!!
!
!!
!!!
!!!
!!
!
!!!!!!!!
!!!!
!!!!
!
!!!
!
!
!!!
!
!!
!
!!!
!
!!
!
!!!!!!!!!
!!
!
!
!
!!!
!!
!!
!!
!!
!
!
!
!
!!
!!!
!!!
!!!!
!!
!
!
!
!!
!
!
!
!
!
!!!!
!
!
!
!
!
!! !!
!!
!!
!
!
! ! !
! !!!
!!
!
!!!!!
!!
!!
!
!
!
!!
!!
!!!
!!!
!!!
!!!!
!
!!
!!
!!!
!!!!!
!!
!
!
!!!!!
!
!!
!!!
!!
!
!!
!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!
!
!!
!!
!
!
!
!!
!
!
!! !!
!!
!!!
!
!
! !!
!
!! !!
!
!!!!
!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
! !!
!!
!!!
!
!!
! ! !
!
!
!!
!
!
!!
!
! !!!!
!
! !
!
!!
!!!!!!
!
!
!
!
!
!!
!
!
!
!!
!
!!
!
!
!
!!
!
!
!
!
!
! !!
!
!!
!
!
!
!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!!
!!
!
!
!!
!
!!
!
!
!
!
!
!!!
!
!
!
!!
!
!!!
!
!
!
!
!
!
!
!!
!
!
!!
!!
!
!!
! !
!
!
!
!
!!
!!
!
!!
!
!
!
!
!
!
!
!
!!!!
!!!
!
!
!!!
!
!!
!!!!!
!
!!!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!!
!
!!!
!!
!
!!!!
!!!
!
!
!!
!!!
!
!
!!
!!
!!!!
!!!!
!!
!
! !
!!
!
!
!!
!
!
!
!! ! !
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!!!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!!!!!
!
! !
!
!
!!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!
!!!!!!!!
! ! ! !!
! !!
!
!
!
!
!
!
!!
!
!!!
!
!!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!! !
!!
!
!!
!
! !
!
!!
!!
!
!
! !
!
!
!!
!!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!!! !
!!
! !!
!
!
!
! !
!!
!!
!!
!
!
!
!!
!!
!
!
!
!!
!! !
!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
!
!
!
!!
!
!
!
!!!!
!!
!!!!
!
!
!
!!
! !
!
!
!
!
!
!
!!
!!!
!
!
!
!
!!!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
! !!
!!
!
!
!!
!
!!
!
!!
!!
!
!
!
!
!
!!!
!
!
!
!
!
!!
!!
!
!
!!
!
!
!!!!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
! !
!
!
!!!
!
!
!
!
!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!!
! !!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!!
!
!
!
!
!
!
!!
!!
! !
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!!!
!! ! !
!
!
! !
!!
!
!
!!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!!
!!
!3 !2 !1 0 1 2 3
!1
01
23
45
1st Principal Component
2rd
Princip
al C
om
ponent
!
!
!
!
!
!
0!500 (168)
501!1000 (216)
1001!1500 (328)
1501!2000 (350)
2001!3000 (97)
3001!3500 (22)
!!
!
!
!
!
!
!
!!!
! !!!
!
!!!
!!!!!!!
!!!! ! !!!
!
!!
!
! !
!!!!!
!!
!
!
!!!!!
!! !
!! !!
!!! !!
!
!!!
!!
!
!!!!
!!
!!
!
!!
!!
!!
! !
!!
!
!!!
!!!
!!!
!!
!!
!
!!!
!
!!
!!!
!!!
!!
!
!!!!!!!!
!!!!
!!!!
!
!!!
!
!
!!!
!
!!
!
!!!
!
!!
!
!!!!!!!!!
!!
!
!
!
!!!
!!
!!
!!
!!
!
!
!
!
!!
!!!
!!!
!!!!
!!
!
!
!
!!
!
!
!
!
!
!!!!
!
!
!
!
!
!! !!
!!
!!
!
!
! ! !
! !!!
!!
!
!!!!!
!!
!!
!
!
!
!!
!!
!!!
!!!
!!!
!!!!
!
!!
!!
!!!
!!!!!
!!
!
!
!!!!!
!
!!
!!!
!!
!
!!
!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!
!
!!
!!
!
!
!
!!
!
!
!! !!
!!
!!!
!
!
! !!
!
!! !!
!
!!!!
!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
! !!
!!
!!!
!
!!
! ! !
!
!
!!
!
!
!!
!
! !!!!
!
! !
!
!!
!!!!!!
!
!
!
!
!
!!
!
!
!
!!
!
!!
!
!
!
!!
!
!
!
!
!
! !!
!
!!
!
!
!
!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!!
!!
!
!
!!
!
!!
!
!
!
!
!
!!!
!
!
!
!!
!
!!!
!
!
!
!
!
!
!
!!
!
!
!!
!!
!
!!
! !
!
!
!
!
!!
!!
!
!!
!
!
!
!
!
!
!
!
!!!!
!!!
!
!
!!!
!
!!
!!!!!
!
!!!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!!
!
!!!
!!
!
!!!!
!!!
!
!
!!
!!!
!
!
!!
!!
!!!!
!!!!
!!
!
! !
!!
!
!
!!
!
!
!
!! ! !
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!!!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!!!!!
!
! !
!
!
!!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!
!!!!!!!!
! ! ! !!
! !!
!
!
!
!
!
!
!!
!
!!!
!
!!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!! !
!!
!
!!
!
! !
!
!!
!!
!
!
! !
!
!
!!
!!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!!! !
!!
! !!
!
!
!
! !
!!
!!
!!
!
!
!
!!
!!
!
!
!
!!
!! !
!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
!
!
!
!!
!
!
!
!!!!
!!
!!!!
!
!
!
!!
! !
!
!
!
!
!
!
!!
!!!
!
!
!
!
!!!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
! !!
!!
!
!
!!
!
!!
!
!!
!!
!
!
!
!
!
!!!
!
!
!
!
!
!!
!!
!
!
!!
!
!
!!!!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
! !
!
!
!!!
!
!
!
!
!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!!
! !!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!!
!
!
!
!
!
!
!!
!!
! !
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!!!
!! ! !
!
!
! !
!!
!
!
!!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!!
!!
!3 !2 !1 0 1 2 3
!1
01
23
45
2nd Principal Component
2rd
Princip
al C
om
ponent
!
!
!
!
!
!
!
0!500 (162)
501!1000 (181)
1001!1500 (207)
1501!2000 (392)
2001!2500 (138)
2501!3000 (88)
3001!3500 (13)
colored by SPECfp rate
1st Principal Component
2nd
Prin
cipa
l Com
pone
nt
!!
!
!
!
!
!
!
!!!
! !!!
!
!!!
!!!!!!!
!!!! ! !!!
!
!!
!
! !
!!!!!
!!
!
!
!!!!!
!! !
!! !!
!!! !!
!
!!!
!!
!
!!!!
!!
!!
!
!!
!!
!!
! !
!!
!
!!!
!!!
!!!
!!
!!
!
!!!
!
!!
!!!
!!!
!!
!
!!!!!!!!
!!!!
!!!!
!
!!!
!
!
!!!
!
!!
!
!!!
!
!!
!
!!!!!!!!!
!!
!
!
!
!!!
!!
!!
!!
!!
!
!
!
!
!!
!!!
!!!
!!!!
!!
!
!
!
!!
!
!
!
!
!
!!!!
!
!
!
!
!
!! !!
!!
!!
!
!
! ! !
! !!!
!!
!
!!!!!
!!
!!
!
!
!
!!
!!
!!!
!!!
!!!
!!!!
!
!!
!!
!!!
!!!!!
!!
!
!
!!!!!
!
!!
!!!
!!
!
!!
!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!
!
!!
!!
!
!
!
!!
!
!
!! !!
!!
!!!
!
!
! !!
!
!! !!
!
!!!!
!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
! !!
!!
!!!
!
!!
! ! !
!
!
!!
!
!
!!
!
! !!!!
!
! !
!
!!
!!!!!!
!
!
!
!
!
!!
!
!
!
!!
!
!!
!
!
!
!!
!
!
!
!
!
! !!
!
!!
!
!
!
!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!!
!!
!
!
!!
!
!!
!
!
!
!
!
!!!
!
!
!
!!
!
!!!
!
!
!
!
!
!
!
!!
!
!
!!
!!
!
!!
! !
!
!
!
!
!!
!!
!
!!
!
!
!
!
!
!
!
!
!!!!
!!!
!
!
!!!
!
!!
!!!!!
!
!!!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!!
!
!!!
!!
!
!!!!
!!!
!
!
!!
!!!
!
!
!!
!!
!!!!
!!!!
!!
!
! !
!!
!
!
!!
!
!
!
!! ! !
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!!!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!!!!!
!
! !
!
!
!!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!
!!!!!!!!
! ! ! !!
! !!
!
!
!
!
!
!
!!
!
!!!
!
!!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!! !
!!
!
!!
!
! !
!
!!
!!
!
!
! !
!
!
!!
!!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!!! !
!!
! !!
!
!
!
! !
!!
!!
!!
!
!
!
!!
!!
!
!
!
!!
!! !
!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
!
!
!
!!
!
!
!
!!!!
!!
!!!!
!
!
!
!!
! !
!
!
!
!
!
!
!!
!!!
!
!
!
!
!!!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
! !!
!!
!
!
!!
!
!!
!
!!
!!
!
!
!
!
!
!!!
!
!
!
!
!
!!
!!
!
!
!!
!
!
!!!!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
! !
!
!
!!!
!
!
!
!
!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!!
! !!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!!
!
!
!
!
!
!
!!
!!
! !
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!!!
!! ! !
!
!
! !
!!
!
!
!!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!!
!!
!3 !2 !1 0 1 2 3
!1
01
23
45
1st Principal Component
2rd
Princip
al C
om
ponent
!
!
!
!
!
!
0!500 (168)
501!1000 (216)
1001!1500 (328)
1501!2000 (350)
2001!3000 (97)
3001!3500 (22)
!!
!
!
!
!
!
!
!!!
! !!!
!
!!!
!!!!!!!
!!!! ! !!!
!
!!
!
! !
!!!!!
!!
!
!
!!!!!
!! !
!! !!
!!! !!
!
!!!
!!
!
!!!!
!!
!!
!
!!
!!
!!
! !
!!
!
!!!
!!!
!!!
!!
!!
!
!!!
!
!!
!!!
!!!
!!
!
!!!!!!!!
!!!!
!!!!
!
!!!
!
!
!!!
!
!!
!
!!!
!
!!
!
!!!!!!!!!
!!
!
!
!
!!!
!!
!!
!!
!!
!
!
!
!
!!
!!!
!!!
!!!!
!!
!
!
!
!!
!
!
!
!
!
!!!!
!
!
!
!
!
!! !!
!!
!!
!
!
! ! !
! !!!
!!
!
!!!!!
!!
!!
!
!
!
!!
!!
!!!
!!!
!!!
!!!!
!
!!
!!
!!!
!!!!!
!!
!
!
!!!!!
!
!!
!!!
!!
!
!!
!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!
!
!!
!!
!
!
!
!!
!
!
!! !!
!!
!!!
!
!
! !!
!
!! !!
!
!!!!
!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
! !!
!!
!!!
!
!!
! ! !
!
!
!!
!
!
!!
!
! !!!!
!
! !
!
!!
!!!!!!
!
!
!
!
!
!!
!
!
!
!!
!
!!
!
!
!
!!
!
!
!
!
!
! !!
!
!!
!
!
!
!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!!
!!
!
!
!!
!
!!
!
!
!
!
!
!!!
!
!
!
!!
!
!!!
!
!
!
!
!
!
!
!!
!
!
!!
!!
!
!!
! !
!
!
!
!
!!
!!
!
!!
!
!
!
!
!
!
!
!
!!!!
!!!
!
!
!!!
!
!!
!!!!!
!
!!!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!!
!
!!!
!!
!
!!!!
!!!
!
!
!!
!!!
!
!
!!
!!
!!!!
!!!!
!!
!
! !
!!
!
!
!!
!
!
!
!! ! !
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!!!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!!!!!
!
! !
!
!
!!
!
!
!!
!!
!
!
!
!!
!!
!
!
!
!
!!!!!!!!
! ! ! !!
! !!
!
!
!
!
!
!
!!
!
!!!
!
!!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!!! !
!!
!
!!
!
! !
!
!!
!!
!
!
! !
!
!
!!
!!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!!! !
!!
! !!
!
!
!
! !
!!
!!
!!
!
!
!
!!
!!
!
!
!
!!
!! !
!
!!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
!
!
!
!!
!
!
!
!!!!
!!
!!!!
!
!
!
!!
! !
!
!
!
!
!
!
!!
!!!
!
!
!
!
!!!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
! !!
!!
!
!
!!
!
!!
!
!!
!!
!
!
!
!
!
!!!
!
!
!
!
!
!!
!!
!
!
!!
!
!
!!!!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!
! !
!
!
!!!
!
!
!
!
!
!
!
!
!
!!!
!!
!!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!!
! !!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!!
!
!
!
!
!
!
!!
!!
! !
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!!!!
!! ! !
!
!
! !
!!
!
!
!!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!!
!!
!3 !2 !1 0 1 2 3
!1
01
23
45
2nd Principal Component
2rd
Princip
al C
om
ponent
!
!
!
!
!
!
!
0!500 (162)
501!1000 (181)
1001!1500 (207)
1501!2000 (392)
2001!2500 (138)
2501!3000 (88)
3001!3500 (13)
colored by SPECint ratecolored by ISA
Performance modelFor each cluster of machines, a performance model is constructed using microarchitecture-independent program characteristics and a genetic algorithm. For a given application of interest, a performance estimation is made by making a weighted harmonic average of the performance numbers of similar benchmarks.
Experimental resultsFor each of the SPEC CPU2000 benchmarks, performance numbers (speedups compared to a reference machine) were estimated for 617 machines divided in 26 clusters (by means of cross-validation). This results in an estimated machine ranking. We evaluate the performance deficiency of the estimated best and top 3 machines for each benchmark.
ammp
applu
apsi
art
bzip2
crafty
eon
equake
facerec
fma3d
galgel
gap
gcc
gzip
lucas
mcf
mesa
mgrid
parser
perlbmk
sixtrack
swim
twolf
vortex
vpr
wupwise
1st Principal Component (X!axis)
0.00
0.05
0.10
0.15
0.20
ammp
applu
apsi
art
bzip2
crafty
eon
equake
facerec
fma3d
galgel
gap
gcc
gzip
lucas
mcf
mesa
mgrid
parser
perlbmk
sixtrack
swim
twolf
vortex
vpr
wupwise
2nd Principal Component (Y!axis)
!0.2
!0.1
0.0
0.1
0.2
0.3
ammp
applu
apsi
art
bzip2
crafty
eon
equake
facerec
fma3d
galgel
gap
gcc
gzip
lucas
mcf
mesa
mgrid
parser
perlbmk
sixtrack
swim
twolf
vortex
vpr
wupwise
3rd Principal Component (Z!axis)
!0.2
!0.1
0.0
0.1
0.2
0.3
0.4
SPECint2000 benchmarks SPECfp2000 benchmarks
Benchmarking systems is commonly done to assess system performance, for example using the well-known SPEC CPU benchmarks (www.spec.org). Performance numbers for each of the benchmarks are reported by various organizations, including system and processor manufacturers, both for research and marketing purposes. Two problems come forward when trying to interpret these performance numbers:• Because of the huge amount of data (speedup numbers for 26 benchmarks for 1000+ machines for SPEC CPU2000), getting valuable insights is hard.• It is unclear which benchmarks are representative for a given application of interest. Using average performance numbers across a set of benchmarks (for example SPEC rates) to determine which system is best suited for a given task can be misleading, and may lead to suboptimal purchase decisions.This work presents a solution for both problems; we analyze published performance numbers of commercial machines to obtain valuable insights and propose a methodology to estimate performance for a given application of interest.
benchmark suite
performance numbers application of
interest
!2 !1 0 1 2 3
!2
!1
0 1
2 3
4 5
!3!2
!1 0
1 2
3
1st Principal Component
3rd
Prin
cip
al C
om
po
ne
nt
2n
d P
rin
cip
al C
om
po
ne
nt
!
!
!
!
!!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!!!!
!
!
!
!!
!
!!
!!!!
!!!!!
!
!!
!
!
!
!!
!
!
!
!
!
!
!!!
!!
!
!
!
!
!!
!
!
!!
!!
!
!
!!
!
!
!
!
!!!!
!!
!!
!
!
!!
!!
!
!
!!
!!
!
!
!
!
!
!
!!
!!
!
!
!
!!
!
!!
!
!
!
!!
!
!!!!!
!!
!! !
!
! !!!
!!!!! !
!!!!!!
!!
!!
!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
Pentium III
Pentium 4 (Willamette, 32!bit)
Pentium 4 (Northwood, 32!bit)
Pentium 4 (Prescott, 32!bit)
Pentium 4 (Prescott, 64!bit)
Pentium 4 (Prescott 2M, 64!bit)
Pentium 4 (Cedar Mill, 64!bit)
Pentium 4 Extreme Edition (Gallatin, 32!bit)
Pentium 4 Extreme Edition (Prescott 2M, 64!bit)
Pentium D (Smithfield)
Pentium D (Presler)
Pentium Extreme Edition (Smithfield)
Pentium Extreme Edition (Presler)
Core Duo (Yonah)
Core 2 Duo (Merom)
Core 2 Duo (Conroe)
Core 2 Extreme (Conroe XE)
Core 2 Extreme (Kentsfield XE)
3rd Principal Component
1st Principal Component
2nd
Prin
cipa
l Com
pone
nt
Some observations:
✦ increasing clock speed is more beneficial for SPECint than for SPECfp✦ the new Core 2 architecture is more targeted to integer workloads than previous systems✦ Core 2-based systems are well suited for cache-intensive workloads
PC1PC2
X
Y
!"#!$
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
!"#!%
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
!"#!&
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
!"#!$'
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
Figure 8: Cumulative average performance deficiency as a function of the number of benchmarks considered when predicting the
best performing top-1, top-3, top-5 and top-10 machines.
Figure 9: Performance deficiency for all benchmarks when predicting the top-3 machines.
averageone modelper ISA-modelper-cluster modelper-machine model
cum
ulat
ive
aver
age
perfo
rman
ce d
efic
ienc
y
0%
1
0%
20
%
30
%
0 5 10 15 20 26
number of benchmarks
estimated best
!"#!$
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
!"#!%
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
!"#!&
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
!"#!$'
!"#$%&'()'$%!*+#,&-.
*"#"/,012%',2%&,3%'4%&)(&#,!*%'5%)1*1%!*6
78
978
:78
;78
7 < 97 9< :7 :=
,2%&,3%(!%'#(5%/4%&'>?@!#(5%/4%&!*/".0%&'#(5%/4%&!#,*+1!%'#(5%/
Figure 8: Cumulative average performance deficiency as a function of the number of benchmarks considered when predicting the
best performing top-1, top-3, top-5 and top-10 machines.
Figure 9: Performance deficiency for all benchmarks when predicting the top-3 machines.
averageone modelper ISA-modelper-cluster modelper-machine model
0 5 10 15 20 26
number of benchmarks
cum
ulat
ive
aver
age
perfo
rman
ce d
efic
ienc
y
0%
1
0%
20
%
30
% estimated top 3
!2 !1 0 1 2 3
!2
!1
0 1
2 3
4 5
32
10
!1!2
!3
1st Principal Component
3rd
Prin
cip
al C
om
po
ne
nt
2n
d P
rin
cip
al C
om
po
ne
nt
!!!
!
!
!!
!!
!
!
!
!!
!
!
!
!
!!!
!!
!
!
!!
!
!
!
!
! !
!
!
!
!!
!
!
! !
!
!
!!
!!
!!
!
!!
!
!
!!!
!!
!
!!!
!
!!!
!
!
!
!
!
!
!
!
!
!!!
!
!
!!
!!!
!
!
!!
!
!!
! !!
!
!!
!
!
!!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!!
!
!
! !
!
!
!
!!
!
!
!
!
!
!
!! !
!
!
!
!!!
!!
!
!!
!!!
!
!
!
!!!
!!!
!!
!!
! !
!
!
!!!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!!
! !
!
!
!
!
!
! !
!
!
!
!
!
!
!
!
!
!
!!
!!
!
!
!
!
! !
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!!
!!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
!
!
!
! !
!
!
!
!!
!
!!
!
!!
!
!
!
! !
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!!
!
!
!!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
! !
!
!
! !!
!
!!
!
!
!
!
!!
!
!
!
!
!
!
!!
!!
!
!!!!!!
!!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!!!!!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!!!
!
!
!
!
!
!
!
! !! !
!
!!!
!
!
!
!
!!
!
!
!
!!
!
!
!
!
!
!
!
!
!!
!
!
!
!!
!
!
!!
!
!!
!
!
!
!
!
!
!!
! !
!
!
!
!
!!!
!
!
!
!
!
!
!
!
!!!
!
!
!
!
!
! !
!
!
!
!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!!!
!
!!
!
!! !!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!!!!
!
!
!
! !
!
!
!
!
!
!
!
!
!
!
!!
!
!
!!
!
!
!
!
!
!
!
!
!
!!
!!!!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!!!
!
!
!
!!
!!
!
!
!
!
!
!
!
!!
!
!
!!
!
!
!!
!
!
!!
!
!
!
!
!!
!
!
!
!!
!
!
!!!!
!!
!!
!
!
!!
!
!
!
!
!
!
! !!!
!
!
!
!
!
!
!
!
!
!
!
!! !
!
!
!
!
!
!
!
!
!
!!
!
!!
!
!
! !!
!!
!!
!
!!!!!! !!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!!
!
!
!
!
!
!!!!!
!!
!!
!
!
!
!
!
!!
!
!
!
!!
!
!
!!!
!
!
!
!
!
!
!!!
!
!!
!
!
!
!
!
!
!
!
!
!!
!
!
!
!
!!
!!
!
!
!
!!!
!
!
!
!
!
!!!
!!
!
!
!!
!
!
!
!
!
!!
!
!
!!!!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!!
!
!!!
!
!
!
!
!
!
!
!
!!
!
!
!!
!!
!
!
!!
!!
!
!
!!
!!
!
!
!
!!!
!
!!
!
!
!
!
!
!!!
!
!
!!
!!
!
!
!!
!
!
!!
!
!
!!
!
!
!
!
!
!
!!!!!
!
!
!
!!
!
!
!
!!
!
!
!
!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!!!
!
!
!!!!
!
!
!
!
!
!!
!!
!
!
!!!!
!!!
!
!
!!!
!
!
!
!
!
!
!
!
!
!!!!
!
!!
!
!
!
!!
!
!!
!
!
!
!
!
!
!
!
!
AMD 64!bit
DEC Alpha
Intel 64!bit
Intel 32!bit
Intel Itanium
MIPS
PA!RISC
IBM PowerPC
Sun SPARC