Simple Steps to Win the Site Performance Battle (and other hopefully interesting stuff) CHRIS WELLS – CEO – NEXCESS.NET LLC SOUTHFIELD, MI USA
Quick Facts About Detroit, MI • It’s the potato chip capital of the world • based on consumption L
• It had the 1st mile of concrete road (1909) • Between 6 and 7 mile roads on M-‐1 (Woodward)
• Home to the only floating Post Office in the U.S. • J.W. Westcott Co (Zip code: 48222)
• It has the most registered bowlers nationwide
• Biggest municipal bankruptcy in the U.S. • Which makes us National Bankruptcy Champions!!
• Eat your heart out Jefferson County, AL!! We did 4 x better than you!!
Notes About This Talk • Lessons learned from MagentoLive Australia • If masses walk out mid-‐talk I’m being:
A) WAY too technical
B) Ridiculously boring
• Lessons learned from MagentoLive UK • If I sweat profusely:
A) DO NOT BE ALARMED
B) Assume the technical folks located at the back of the room simply swapped out the real talk I had prepared with something they thought was more fun and I’m simply dying a slow 35 minute death on stage before your very eyes.
The Basics NOT On Today’s Agenda • OS level set up / configuration
• Apache or Nginx
• Magento caching
• Magento EE’s FPC
• Varnish / Turpentine or other reverse proxy caching techniques
• Other performance extensions
• At this point we’re all well aware…
• So let’s dig a little deeper into some fun stuff!
*ON* Today’s Agenda • PHP – 92%
• MySQL – 3% (if time permits)
• Drives – 5% (if time permits)
• If you leave here today with some food for thought then I’m happy!
PHP – Personal Home Page • Yes, I know.. It’s now “PHP: Hypertext preprocessor”
• Magento used to be version locked to PHP <= 5.3
• As of January Magento officially supports PHP 5.4! Woohoo! • You may need to patch your Magento for it to work with 5.4
• So…?
• PHP 5.4 is a good deal faster than 5.3 • Empty hash table optimizations
• Literal tables • Interned strings • Zend Engine VM tuning
• But what does this mean for Magento?
Notes On Magento • Magento is very CPU heavy • The bulk of this heaviness is within PHP code
• MySQL CPU usage is typically bound by PHP’s CPU usage (speaking in single server terms)
• Magento is not very I/O heavy (contrary to popular belief) • Again, I/O is bound by PHP’s CPU usage
• More on I/O later
• Quantitative notes to follow J
Let’s Test Us Some PHP • 1-‐hour Siege runs (meant to tax PHP, not MySQL)
• EE tests • Siege of 20,000 SKUs hitting ~20,000 random URLs
• FPC enabled (Redis)
• CE tests • Siege of 120 SKUs hitting ~120 URLs.
• Bare metal configuration of: • Dell R420 – Dual E5-‐2420 6-‐core • H710 512MB NV + 4 x 15K 300GB RAID 10
• 24G RAM
• Ok ok, let’s see what we’ve got!
PHP 5.3.24 (CE Front-‐end Baseline)
0"
20"
40"
60"
80"
100"
120"
140"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
~42 t/sec
PHP 5.3.24 vs. PHP 5.4.28 (CE Front-‐end)
0"
20"
40"
60"
80"
100"
120"
140"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
0"
20"
40"
60"
80"
100"
120"
140"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
PHP"5.4.28"
~52 t/sec (~23% increase!)
Magento cache population
PHP 5.3.24 (EE Front-‐end Baseline)
0"
100"
200"
300"
400"
500"
600"
700"
800"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
~550 t/sec
GO GO FPC!
PHP 5.3.24 vs. PHP 5.4.28 (EE Front-‐end)
0"
100"
200"
300"
400"
500"
600"
700"
800"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
0"
100"
200"
300"
400"
500"
600"
700"
800"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
PHP"5.4.28"
~615 t/sec
(~12% increase!!)
FPC / Magento cache population amplifies base PHP version improvements
PHP 5.3.24 vs. PHP 5.4.28/.14 (EE Front-‐end)
0"
100"
200"
300"
400"
500"
600"
700"
800"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.3.24"
PHP"5.4.14"
PHP"5.4.28"
Even upgrading from 5.4.14 shows gains of ~4%
PHP 5.3.24 vs. PHP 5.4.28 (Re-‐index from CLI)
0" 10" 20" 30" 40" 50" 60" 70" 80" 90"100"
PHP"5.3.24"
PHP"5.3.24"
Time In Seconds
86#88#
90#92#
94#96#
98#100#
PHP#5.3.24#
PHP#5.4.28#
~8% improvement
PHP 5.4.28 vs. PHP 5.5.12 (EE/Front-‐end)
0"
100"
200"
300"
400"
500"
600"
700"
800"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.4.28"
PHP"5.5.12"
~781 t/sec
~25% better than 5.4 ~41% better than 5.3
WOW J
FPC / Magento cache population again
amplifies base PHP version improvements
Notes On PHP 5.5.x+ • !! Warning – Magento does not yet officially support 5.5+ !!
• APC goes out • Magento configs may need to be tweaked if you’re using APC’s key/val caching
• Some extensions will need to be updated / re-‐coded for sure
• OPcache is introduced • OPcache is the name of the bundled ZendOptimizer+ opcode caching system
• Seems to work out of the box without too much fuss
• More research is needed here – was very surprised with the performance results
• !! Warning – Magento does not yet officially support 5.5+ !!
PHP 5.4.28 vs. PHP 5.5.12 (Re-‐index from CLI)
85#86#
87#88#
89#90#
91#
PHP#5.4.28#
PHP#5.5.12#
Time In Seconds
~4% improvement
PHP 5.5.12 vs. PHP 5.6.0-‐BETA2 (EE/Front-‐end)
0"
100"
200"
300"
400"
500"
600"
700"
800"
1" 2" 3" 4" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 20" 25" 30" 35" 40" 45" 50" 55" 60"
Siege"tran
sac6on
s"/"se
cond
"
Dura6on"(in"minutes)"
PHP"5.5.12"
PHP"5.6.0"(BETA"2)"
L Essentially equal within
margin of error
PHP 5.5.12 vs. PHP 5.6.0-‐BETA2 (Re-‐index from CLI)
72# 74#76#
78#80#
82#84#
86#88#
PHP#5.5.12#
PHP#5.6.0#BETA2#
Time In Seconds
BUT, we got ~11% improvement on the re-‐index!
~21.5% better than 5.3.24!
Notes on PHP-‐NG • Removes numerous heap allocations (and de-‐allocations)
• Stores more native data directly on the stack
• Removes the need to garbage collect basic primitives (bool, long, etc)
• PHP’s reported Wordpress benchmarks show very good results • 26.75 sec -‐> 18.95 sec (36% improvement)
• 9.5M instructions -‐> 5M internal instructions executed (90% reduction)
• Take some comfort in knowing that more gains are on the way from PHP folks directly
MySQL – “My SQL” • My is the Michael Widinius’s daughter’s name (this was news to me)
• Lots of “brand” options on the MySQL front unlike PHP • MariaDB
• Percona Server / Percona Cluster • Oracle’s MySQL
• Equally many options on the storage engine front • MyISAM
• InnoDB • XtraDB
Let’s Test Us Some MySQL • ~1 hour re-‐index runs (ideally meant to put some load on MySQL)
• Full CLI-‐based re-‐index
• Tested latest EE with various versions/brands of MySQL
• DB contained 800,000 SKUs (thanks Magento performance team)
• Bare metal configuration of: • Dell R420 – Dual E5-‐2420 6-‐core • H710 512MB NV + 4 x 15K 300GB RAID 10
• 24G RAM
• Ok ok, let’s see what we’ve got!
Magento CLI Re-‐index (EE)
Time In Seconds
3100$ 3150$ 3200$ 3250$ 3300$3350$
3400$3450$
3500$3550$
MySQL$5.1.71$
Percona$Cluster$5.5.34$
MariaDB$10.0.10$
MySQL$5.6.17$
Percona$Server$5.6.17$
Magento CLI Re-‐index (EE)
Time In Seconds
0"5"
10"15"
20"25"
Percona"Server"5.6.17"
MariaDB"10.0.10"
MySQL"5.1.71"
Percona"Cluster"5.5.34"
MySQL"5.6.17"
Time"Spent"in""sys""
Thoughts On The Tests • Re-‐index runs, while DB intensive, are still PHP heavy
• Oracle’s MySQL did surprisingly well • If concurrency were higher I’d expect Percona/MariaDB to excel
• DB bound writes (beyond the re-‐index) were missing • This would tax the DB servers considerably
• TCP (and other) overhead for Percona Cluster doesn’t seem to kill performance
Hard and Solid State Disk Drives • Hard drive vs. disk drive vs. solid state drive vs. SAS, etc. • IOPs (I/O operations / sec) • More is better, but don’t throw money away
• All SSDs not created equal • Don’t believe quoted specs! • Don’t buy consumer grade SSDs!
• Quality SSDs are ~2x the cost of quality HDD of same size
• Know your workload profile!
Live Client Case Study • Magento EE 1.12.0.1
• ~700,000 SKUs
• 350+ orders / hour
• 15 x web servers (bare metal) • Dell R420, Dual E5-‐2450 8-‐core, 24GB RAM
• 2 x DB servers (replicated) – peak @ ~20,000 qps
• 1 x file server (shared storage for media)
• FPC / Magento cache in Redis
• ~1 week sample size
Database Server IO Requirements
Read Write
Average IOPs: 3.66 471
Min IOPs: 0.0 6.87
Max IOPs: 830 2250
Standard deviation: 34.5 197.0
95% below: 60.4 795
99.999% below: 151 1310
• Percona Server 5.5.35 • 128GB RAM / 60GB MySQL DB (innodb_buffer_pool = 75G)
• 8 x 300GB 15k HDD/SAS RAID 10 -‐ H700/1G RAID card • Replicating to a hot-‐standby DB server (unused by Magento)
???
File Server IO Requirements
Read Write
Average IOPs: 13.7 7.65
Min IOPs: 5.58 2.97
Max IOPs: 85.8 186
Standard deviation: 6.72 11.0
95% below: 24.7 25.8
99.999% below: 42.3 54.6
• Serving /media and /var only over NFS
• 24GB RAM -‐ 600GB data on disk
• 4 x 600GB 15k HDD/SAS (RAID 10) -‐ H700/512M RAID card
Notes On Drives • Get quality drives regardless of technology • No consumer SSDs!
• Caches are very good at relieving disk-‐bound I/O • MySQL query cache
• Redis/memcaches
• RAID card caches • On-‐disk controller caches
• Invest in CPU over IOPs unless you can afford both
• Magento is typically bound by CPU, not I/O – Know your workload profile