Latest ESRP V3.0 results · 2013. 2. 28. · last time. Specifically, the IBM XIV 2810 (#2), the...

© 2013 Silverton Consulting, Inc. Page 1 of 5 All Rights Reserved

This Storage Intelligence (StorInt™) dispatch covers Microsoft Exchange 2010 Solution Review Program (ESRP) v3.0 results.1 Fortunately, over the last quarter the there have been at least 14 new submissions in this over 5,000 mailbox category and 23 in all categories but I have lost count how many new submissions since we last discussed this category in January of 2012. As such, just about every chart we report on has some changes.

Latest ESRP V3.0 results We start our discussion with our normalized database transfers per second per 1000 mailboxes.

Figure 1 ESRP v3.0 normalized DB xfers/sec/1Kmbx for over 5K mailboxes

As can be seen above there are four new submissions that ranked in the top ten since last time. Specifically, the IBM XIV 2810 (#2), the HDS HUS 110 Dynamic Provisioning (#6), the HDS AMS2300 Dynamic Provisioning (#7), and the HDS USP-V Dynamic Provisioning (#8). Recall that this is a normalized metric (using 1000 mailboxes) for the denominator. As such, it tries to account for the fact that larger mailbox counts often generate more database transfers/second (all things being equal).

1 ESRP results from http://technet.microsoft.com/en-us/exchange/ff182054.aspx, as of 27 February 2013 Silverton

Consulting, Inc. StorInt™ Dispatch

ESRP/Jetstress Performance SCIESRP130227-010

© 2013 Silverton Consulting, Inc. Page 2 of 5 All Rights Reserved twitter.com/RayLucchesi|RayOnStorage.com

+1-720-221-7270|SilvertonConsulting.com

The only other caveat for our normalized DB/xfers/sec is that Jetstress 2010 parameterizes its IO activity, which in this case ranges anywhere from 0.80 down to 0.18 IO/sec/mbx. Only the Exchange gods can fathem why this rate should vary so much for this category but it has a direct bearing on rankings for this and other charts. For instance both of the top two systems used a rate of 0.8 and the next highest rate was a 0.6 used for #6. One would almost have to normalize again for projected IO rate to better portray system performance. Next we turn to Exchange Log playback.

Figure 2 ESRP Log playback for over 5000 mailboxes In Figure 2, the only new submission is the Infortrend EonStor at #9. All the others were on the previous version of this chart. In the past Infortrend storage has been OEMed from (NetApp) Engenio. I am not sure whether this subsystem is another OEM but it could very well be. We have always liked log playback because it doesn’t correlate well with Jetstreess’s IO rate (see above). However, it may have some relationship with number of disk drives in the system. For instance, drive counts for this top ten vary from a low of 16 (#9, Infortrend) to a high of 512 (#8, HDS USP-V) and, #1 used 96 drives while #2 used 336 drives. So, maybe the correlation between log playback time and drive count is not as strong as I suspect. Not sure how to characterize log playback activity. It is processing database log records and applying them to the various databases so it’s a combination of sequential activity (for the log) and random updates for the databases. Previously I thought log playback performance




might be correlated to the structure of the Exchange databases and their spread across Exchange servers. And this may indeed be the case because the #1 HP EVA submission had 4 (2 active and 2 passive) Exchange servers for 6000 mailboxes, which seems a bit much. Next we review total database backup throughput.

Figure 3 ESRP Total database backup (MB/sec) for over 5K mailboxes

Figure 3 shows another computed metric that represents the total Exchange database backup throughput, across all servers and all databases for a storage subsystem. ESRP reports database backup throughput on a per database or per Exchange server level. But I find from a pure storage perspective it’s better to understand backup throughput for the whole storage system. In Figure 3 above there are five new submissions, four from IBM, XIV (#1&2), DS5300 (#9) and DCS3700 (#10). I believe the DS5300 and DCS3700 are OEMed solutions. The only other new submission here is the HDS USP-V (#4) using 32Kmbx. I view database backup as essentially a highly intensive sequential throughput activity. The fact that XIV Gen3 demonstrated such high throughput here (~12.7 GB/sec) is impressive. Their #2 submission had less disk drives but added SSDs to compensate, but in this case didn’t necessarily perform as well. Next we analyze database transfers per second on a spindle basis.




Figure 4 ESRP Scatter plot DB/xfers/second against spindle count Finally, in Figure 4 we show the correlation between spindle count and un-normalized database transfer per second. The regression coefficient is not that good at ~0.5 and there are some significant outliers, especially as they get over ~250 drive spindles. One caveat here is that we eliminate any systems that use NAND as cache or SSD from this chart. As such, this chart shows disk-only storage system submissions. We often hear arguments that benchmarks are just a game, where he who throws the most hardware at it wins. The correlation above refutes this notion. As long as one assumes that the number of drives is a reasonable surrogate for the amount of hardware used in a submission. Yes it’s missing SSD or FlashCache systems and yes it’s still impacted by Jetstress IO rates, but assuming the former is minimal and the later is set to show the system at its best, then we have a reasonable data set to understand if more hardware wins. The answer, it appears, is not always.




Conclusions I thought we had seen the last upsurge in ESRP submissions but this past 3 months saw an avalanche. I now believe Microsoft’s upcoming rollout of a new Exchange (sometime in 2013) is forcing all old submissions to be published. Reiterating sentiment from previous reports, there are more mysteries in ESRP reports than in any other performance results we examine. Also, ESRP/Jetstress storage system performance seems designed to be difficult to compare but in our view, merit the effort. Indeed, Microsoft would state that ESRP is not a benchmark at all but we feel it’s as true a view of storage performance as any we examine. Please, feel free to contact us with any constructive ideas on how to improve our ESRP analysis. We are always open to change our views if that will lead to a clearer picture of system performance. In that regard, our contact information can be found in the footer below or on our website at SilvertonConsulting.com. For a more complete explanation of ESRP and for that matter SPC-1 and SPC-2 block storage performance see our recently updated SAN Storage Buying Guide available for purchase on our website.2 Alternatively, you can use the QR code below left, which should point you to the right webpage to purchase the guide. If you received this report from someone else, please signup for our free monthly newsletter using the QR code below right to receive your own copy. We analyze storage system performance results/benchmarks each month and will analyze Microsoft ESRP results again in three months.

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community

(QR code: SAN Storage Buying Guide) (QR code: Newsletter signup)

2 Please see http://silvertonconsulting.com/cms1/products/san-storage-briefing/

Date post:	03-Feb-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Latest ESRP V3.0 results · 2013. 2. 28. · last time. Specifically, the IBM XIV 2810 (#2), the...

Documents