1 © Copyright 2011 EMC Corporation. All rights reserved.
Microsoft SharePoint 2010 Infrastructure Best Practices
EMC Global Solutions
2 © Copyright 2011 EMC Corporation. All rights reserved.
• SharePoint architecture and performance factors
• SharePoint farm virtualization
• Storage best practices – Sizing and performance
• Remote BLOB Storage
• EMC Storage options and features
• SharePoint farm Protection
Agenda
3 © Copyright 2011 EMC Corporation. All rights reserved.
Proven Solutions approach
Capture
& Define
Test and Validate
Document Publish
Singapore
Shanghai,
China
Cork, Ireland
Hopkinton, MA Santa Clara, CA
Vienna, Austria
1 2 3 4
4 © Copyright 2011 EMC Corporation. All rights reserved.
Server
‒ CPU
‒ Memory
‒ HBA/CNA
‒ NIC
BLOB
Storage
(Optional)
SharePoint performance - The user experience
Storage
− Content/Metadata
− Search
− System
Network Client
Document
Request Web Front End
SQL Server
BLOB
Retrieval/Creation
Domain
Controller
Technet: http://technet.microsoft.com/en-us/library/cc287790(office.12).aspx
Type Of
operation Examples
Acceptable
user response
time
Common • Browsing to the home page
• Browsing to a document library <3 seconds
Uncommon • Creating a subsite Creating a list
• Uploading a document to a document library <5 seconds
Rare • Backing up a site
• Creating a site collection <7 seconds
Authentication
5 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint Farm Topologies
Web
Application
Database
Small Medium Large Scale out approach = More servers ?
H/W Eval Prod
RAM 4GB 8GB
CPU 4 Cores
H/W Small Medium
RAM 8 GB 16 GB
CPU 4 Cores 8 Cores
Web/Query
All DBs
App
Web
Query/Crawl
Search DBs SharePoint DBs
App
Web Servers Groups
Query Crawl
Search DBs SharePoint DBs Content DBs
User requests Crawling/Admin
App Servers Groups
Central Admin
/Office/Other
H/W Eval Prod
RAM 4GB 8GB
CPU 4 Cores
http://technet.microsoft.com/en-us/library/cc262485.aspx http://technet.microsoft.com/en-us/library/cc298801.aspx
6 © Copyright 2011 EMC Corporation. All rights reserved.
Key Benefits – Virtualizing SharePoint
• Consolidation – Achieve 2-10x consolidation ratio, especially for
larger deployments
• Performance – Improved front end performance with more, smaller
WFEs rather than few large WFEs.
• Availability – VM based protection for SharePoint provides
homogeneous high availability (VMware HA, WFC).
• Business Continuity - Simplified DR management (vCenter Site
Recovery Manager, Cluster Enabler)
• Maintenance – Live migration of virtual machines (VMware vMotion,
Hyper-V Quick/Live Migration)
• Load Balancing – Maximized overall performance with balanced HW
utilization across the farm (VMware DRS, SCVMM PRO)
7 © Copyright 2011 EMC Corporation. All rights reserved.
Virtualizing Server Roles In SharePoint
Application (Excel, Doc Conv, etc)
Index/Crawl
SQL
Web Front End / Query
CPU – Application dependent
Scaling out is more efficient
CPU – User concurrency, Search requests
Scaling out is more efficient
Network – segment vNICs and vSwitches
Redundant (Non redundant in MOSS 2007)
CPU – Crawling, indexing (depends on content type/size)
Scale out (Up only with MOSS 2007)
Memory intensive
CPU – Document updates, Search, Backup
VMFS/RDM, VHD/Pass-Through
Scale up/out (Hyper-V ≤ 4 vCPU, VMware ≤8 vCPU)
Failover Clustering, Mirroring, VMware HA
Server Roles/Priority What To Consider
4th
3rd
2nd
1st
Understanding your existing workload (WFE to SQL) and requirements is better than
any general best practice!!!
8 © Copyright 2011 EMC Corporation. All rights reserved.
A Day in the life of SharePoint… SQL Server CPU
The majority of load comes from systematic operations…
Sample anonymous customer data SharePoint Timer Job Reference - http://technet.microsoft.com/en-us/library/cc678870.aspx
9 © Copyright 2011 EMC Corporation. All rights reserved.
A Day in the life of SharePoint… SQL Server Storage I/O
Sample anonymous customer data
Plan for user load peaks, not systematic peaks…
10 © Copyright 2011 EMC Corporation. All rights reserved.
Virtualized SharePoint - Reference Architecture
Role Qty VM specs
VMware vSphere 4.0 ESX servers (physical) 3 Nodes 4-socket quad-core (16 cores), 48 GB RAM
SQL Server 2008 2 8 vCPU, 32 GB RAM
Web front ends 6 4 vCPU, 8 GB RAM
Application servers (Incl. Central Admin) 2 2 vCPU, 4 GB RAM
Search Crawl Servers 2 4 vCPU, 8 GB RAM
Search Query Servers 2 4 vCPU, 8 GB RAM
Domain Controllers 2 2 vCPU, 4 GB RAM
• 1 TB User content
• 10 DBs/Site Collection 100GB each
• 110 GB data files on 130 GB R5 LUNs
• 9 GB log files on 10 GB LUNs
• 100GB R1/0 LUNs for Search DBs
(Property & Crawl Stores)
• R5 LUNs for Crawl Index
Maximum capacity achieved at 10% concurrency = 20,490 Users
11 © Copyright 2011 EMC Corporation. All rights reserved.
System and Configuration Shared Service Applications Content
SQL Server
SharePoint 2010 Storage Elements
Configuration Databases
Central Admin
tempdb, master ,model , msdb
Usage & Health Data Collection - Logging
Search – Admin, Property, Crawl
Web Analytics – Staging, Reporting
User Profiles – Profile, Synchronization, Social Tagging
Managed Metadata- Term Store
State
Business Data Connectivity
Secure Store
Search Index
Index Partition/s Query Component/s
Service Application Data
System Volumes
Boot/OS/VMs Web Parts & Features SharePoint Binaries
SA Volumes
Content Databases
BLOB Store
12 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint 2010 Storage I/O Characteristics
I/O Type Content
Databases tempdb
Search
Databases (property & crawl stores)
Index
Component
Query
component
Read (KB) 16 8 8 32 32
Write (KB) 16 32 16 64 64
* Based on average workloads in a collaboration farm (Browse/Search/Modify – 80/10/10)
SQL Server Search Servers
R:W Ratio 95:5 50:50 60:40 90:10 70:30
SQLIO/SQLIOSim are I/O stress tools - should not be considered as “performance requirements” !
13 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint 2010 Storage performance Requirements/estimates for Search
Based on a Microsoft case study – Mileage may vary!!!
Database Role Microsoft’s Estimate IOPS Typical averages observed
Crawl Database 3,500 – 7,000 (R70:W30) 2,000
Property Database 2,000 (R30:W70) 600
Query Component 2,000 per Active/Failover pair
(Load/Write/Merge)
450
Crawl Component 300-400 IOPS 80-100
http://technet.microsoft.com/en-us/library/cc298801.aspx
Crawler Query SQL Server
Crawl Component Query Component Crawl
DB
Property
DB Search Admin
DB
14 © Copyright 2011 EMC Corporation. All rights reserved.
Reference Architecture – Storage Layout
Cost driven configuration
‒ 13,000 Heavy SharePoint users
‒ 1 TB User content with RBS FILESTREAM Externalization
‒ Protected with EMC RecoverPoint
SATA Disks
‒ RAID5 –VM Boot Luns, Content Databases
‒ RAID10 – Search Databases, TempdDB,
‒ FAST Cache to compensate for disk latencies
Fibre Channel
‒ RAID10 – RecoverPoint LUNs (Target & Repository)
15 © Copyright 2011 EMC Corporation. All rights reserved.
Storage Role
% of
Corpus
Size
Typical sizes GB Recommended
RAID
Virtual Machine Boot Volumes - 80 R-5
Application Volumes - 50 – 300+ R-5
Content
Databases
Data File Volume - 100 – 200 R-5
Log File Volume 10% of Data
File 10 – 20 R-5 / R-10
tempdb
Data File Volume 10% 100 – 300 R-10
Log File Volume 2% 10 – 100 R-10
SharePoint Storage Sizing Volume Sizing
Storage and SQL Server capacity planning and configuration http://technet.microsoft.com/en-us/library/cc298801.aspx
Hardware and software requirements http://technet.microsoft.com/en-us/library/cc262485.aspx
16 © Copyright 2011 EMC Corporation. All rights reserved.
Storage Role % of
Corpus Size
Sample size
(1TB Content)
Recommend
RAID FASTCache?
SQL Search
Databases
Crawl Store DB 0.046 × (sum of
content databases) 46GB R-10 Yes
Property Store DB 0.015 × (sum of
content databases) 15GB R-10 Yes
Search Server
Volumes
Index Component 10% 100GB R5 / R10 No
Query Component 10 – 35% * 2.85 150 – 1TB R-5 / R-10 No
SharePoint
Configuration
Databases
- Data & Log
Volume
SharePoint_Config - 5GB R-5 No
SP_AdminContent - 15GB R-5 No
Usage and Health Data
Collection Database
- (based on %
monitoring)
50 -
500GB(?) R-5 No
SharePoint Storage Sizing Volume Sizing
SharePoint 2010 Database sizing characteristics
http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
17 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint Storage Best Practices SQL Server – Storage Allocation
Use 64KB unit allocation size when formatting a DB Volume
Plan Database file sizes accordingly – Don’t rely on autogrowth
File growth can cause locking, set files size and autogrowth increments appropriately
– Using RBS would keep the SQL Database files small
When using Thin/Virtual provisioning – Use the “Quick Format” option
– Enable Instant file initialization* Enhances the speed for data file creations, restores, data file growth
Assign SQL service account to “Perform Volume Maintenance Tasks” permission
– Log files are fully allocated and zeroed upon creation or growths
* http://msdn.microsoft.com/en-us/library/ms175935.aspx
18 © Copyright 2011 EMC Corporation. All rights reserved.
* Hard to interpret due to virtualization of storage. Consider in combination with response times
SharePoint Storage Best Practices SQL Server - Performance
Data file Latency
Read/Write Operations
Log file Latency
Write Operations
Recommendation
< 10 ms < 5 ms Very Good
< 20 ms 5 – 10 ms Acceptable
> 20 ms > 15 ms Investigate and Improve
Important Perfmon I/O counters Real Meaning!
Average Disk/sec Read or Write Disk Latency
Current Disk Queue Length* Outstanding I/Os
Disk Reads/Writes per Second IOPS
Average Disk Bytes/Read & Write I/O Size (KB)
Plan for optimal disk response times
19 © Copyright 2011 EMC Corporation. All rights reserved.
• Should I use Thin Provisioned Pools on VMAX?
‒ Yes, reduce initial storage requirement
‒ Thin device performance is equal to thick on VMAX
• Is VMAX FAST VP a good fit for SharePoint 2010?
‒ Yes, but for maximum efficiency, it depends on which storage role
‒ Search Index component ? No ‒ Highly changing, throw-away data
‒ Search Query component ? Yes ‒ Highly-read data with small burst write changes
‒ TempDB? Yes ‒ The same blocks are re-used on disk and performance of TEMPDB directly affects
SharePoint performance request - tempdb is used in every SharePoint request
‒ Helps to handle unanticipated performance requirements
EMC Storage Technologies for SharePoint 2010 Symmetrix VMAX
20 © Copyright 2011 EMC Corporation. All rights reserved.
• Should I use Thin provisioning with CX/VNX for SharePoint? − In most cases, yes
• How can SharePoint benefit from FAST Cache? − Performance boost, mainly for Search
EMC Storage Technologies for SharePoint 2010 VNX/CX
W/O FAST Cache FAST Cache
Maximum user capacity 12,240 13,080
RPS 20.4 21.8
Crawl operations 45 items/second 86 items/second
Browse - Avg. User Response time (sec) 2.42 2.31
Search – Avg. User Response time (sec) 1.37 1.0
Storage Role LUN
Compression?
Thin
Provisioning?
Index/Query Component Storage No No
SQL Search Database No No
SQL TempDB No Maybe
SQL Content Databases No Yes
BLOB Store Yes Yes
21 © Copyright 2011 EMC Corporation. All rights reserved.
• “BLOB” = Binary Large Object
– Terminology used to the format of storing a “file” within a
SharePoint or a SQL Database
• RBS = Remote BLOB Storage
– A method of externalizing that BLOB to an NTFS Volume
• Reduces storage footprint for SQL Server
• Enables better storage tiering (TCO)
Remote BLOB Storage
Information on RBS: http://technet.microsoft.com/en-us/library/ee748638.aspx
WFE SQL Server Content
Database
R:\ RBS Tier
0x897d1786e9ab98dvw39
22 © Copyright 2011 EMC Corporation. All rights reserved.
Storage tiering and SharePoint
EMC partners with Metalogix for BLOB externalization (EMC Select)
– RBS can reside on Block/File/Object (VMAX, VNX, Isilon, DataDomain)
– Atmos/Atmos VE - Cloud Optimized Storage
– Centera API
Performance improvement for large objects retrieval mainly (>1Mb)
SharePoint 2GB file size limit is not lifted by using RBS
Feature SQL Server
RBS FILESTREAM StoragePoint
Externalization using SMB/CIFS/NFS protocols No (Local volumes only) Yes
Multiple storage endpoints per content database No Yes
Use SharePoint filename or folder on offloaded BLOBs No (GUID based) Yes
User Interface Powershell Central Admin
BLOBs tiering based on age/size/version policies No Yes
Orphaned BLOB Garbage Collection Basic (RBS Maintainer) Policy-based
23 © Copyright 2011 EMC Corporation. All rights reserved.
Considerations for Remote BLOB storage
Replication, Backup and Recovery
– Native/Item level backup (stsadm based) would include BLOBs
– SQL based backup would only protect the content database metadata
– To maintain consistency: Backup – First Content Databases then BLOB Store
Restore – First BLOB Store then Content Databases
– For DR purposes always tie RBS volumes with SQL Server volumes
– For faster recovery, consider larger intervals of garbage collection jobs (Keeps previous BLOB versions)
Block, File or Object storage?
– Performance: Block would be faster but RBS has typically low I/O
– Storage Efficiency – Block-level LUN Compression – increases storage efficiency, may affect backup performance
– Filesystem-Deduplication – better performance and increased dedup rates
– Object based storage (Atmos/Centera) is viable
http://www.emc.com/collateral/hardware/white-papers/h7048-sharepoint-blob-clariion-atmos-wp.pdf
24 © Copyright 2011 EMC Corporation. All rights reserved.
Reference Architecture - BLOB Externalization
• Farm Profile – Total content ~1TB
– 4.4M documents
– Avg. File Size - 220K
– 1 Site Collection/10 Sites
• Storage Profile – EMC VNX5300
– SQL Database: 15K SAS
– BLOB Store: 7.2K NL-SAS
– CIFS Share for RBS
• Results – Max user capacity – 8,630 (10%)
– BLOBs consumed 92% of content databases
– Full crawl duration – 34 hours
– 20% capacity saved with RBS File system de-duplication
25 © Copyright 2011 EMC Corporation. All rights reserved.
Protecting SharePoint to Enterprise Scales Replication Management for Microsoft SharePoint 2010
Rapid full farm protection while enabling item-level recovery….
– Hardware VSS-based coordinated SharePoint
replication, Enabling farm-wide consistency
– Negligible impact to farm performance even
during the first synchronization
– Utilizing EMC storage replication (Snaps/Clones)
– Simple, intuitive SharePoint discovery and
application set configuration (configure
protection in <8 minutes)
– Works with SQL RBS FILESTREAM
– Full farm restore include search index
consistency on recovery
26 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint Replication Management Reference Architecture
Enabled by EMC Replication Manager, Kroll Ontrack PowerControls
• Hybrid farm (Physical/Virtual)
• 1.5 TB of content (6,818,250 files)
• 15 SharePoint content DBs
• Both SnapView Snaps
and Clones used
SharePoint action SnapView STSADM
Full farm backup (2.5 TB) Clone: 3hr 11min
Snapshot: 9min
39hr 30min
(14.8 MB/s)
Daily incremental SharePoint backup
(~1% daily change rate) Clone: 11min Not tested in this environment
Content database recovery (online) 7min 3hr 12min
(12 MB/s)
Item-level recovery 10min Not tested in this environment (requires recovery farm)
27 © Copyright 2011 EMC Corporation. All rights reserved.
SAN
RecoverPoint
SRDF
MirrorView
VPLEX
SQL
DB Mirroring
Log Shipping
Point in
Time
RM
Business Continuity for SharePoint 2010:
Options
Stretched Farm (Partial Replication)
Mirror Farm (Partial Replication)
Virtualized Farm (Complete Replication)
28 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint 2010 Disaster Recovery: Stretched Farm (Partial Replication)
Application Server/s
WFE/Query Servers
Application Server/s
Crawl Server/s
SharePoint Farm A Secondary Site
WFE/Query Servers
Crawl Server/s
SQL Cluster
Primary Site
EMC Cluster Enabler
Database Mirroring (Synchronous) SQL Cluster
• One logical farm
• SharePoint distribution of web,
search and application
components
• NLB solution for client traffic
control
• Active/Active
• Automated Failover
• Limited to 1ms latencies (SQL-
WFE), min 1GigE
Central Admin
Services DBs
Content
Crawl
Configuration
OS
Query Component
1
Index Partition 1
OS
Crawler
OS
BLOB Store
OS
Query Component
1-Mirror
Index Partition 1
Central Admin
Services DBs
Content
Crawl
Configuration
OS
Crawler
OS
BLOB Store
Witness
29 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint 2010 Disaster Recovery: Mirror Farm (Partial Replication)
Application Server/s
WFE/Query Servers
Application Server/s
Crawl Server/s
SharePoint Farm A
Secondary Site
WFE/Query Servers
Crawl Server/s
SQL Cluster
Primary Site
RecoverPoint
SRDF
VPLEX SQL Cluster
SharePoint Farm B
• Two independent farms
• Content replication only
• Independent search
• Consistency grouping for BLOB
and metadata (SAN)
• Remote snaps/clones to enable
RW copies
• Requires refreshing configuration
regularly
• Long recovery time (RTO)
Central Admin
Services DBs
Content
Crawl
Configuration
OS
Query Component
1
Index Partition 1
OS
Crawler
OS
BLOB Store
OS
Query Component
1-Mirror
Index Partition 1
Central Admin
Services DBs
Content
OS
Crawler
OS
Crawl
Configuration
BLOB Store
30 © Copyright 2011 EMC Corporation. All rights reserved.
Automated DR
Consistency Group
(RP/SRDF/MV) LUNs Grouping SRM CE
Web Front Ends Boot + Query (optional) Protection
Group Cluster Group
Query Servers Boot + Query Component Protection
Group Cluster Group
Index Servers Boot + Index Component Protection
Group Cluster Group
Application Servers Boot + Application Volumes Protection
Group Cluster Group
SQL Server(s)
Boot + Pagefile (optional)
Protection
Group Cluster Group
SQL System Databases
Configuration Databases
Search Databases
Content Databases
RBS BLOB Store
Designing DR consistency for SharePoint
Search consistency Externalized Content consistency
31 © Copyright 2011 EMC Corporation. All rights reserved.
SharePoint 2010 Disaster Recovery: Virtualized Farm (Full Replication)
SharePoint Farm A
Secondary Site Primary Site
BLOB Store
VMs
RecoverPoint
SRDF
VPLEX
VMs
VMs
VMs
• Resource/Protection Group level
granularity
• Active/Active (Sync distances) or
Active/Passive (Async distances)
Failover automation:
VMware Site Recovery Manager (SRM) Protection Groups for all server roles
EMC Cluster Enabler with Hyper-V failover clustering
Databases
BLOB Store
VMs
VMs
VMs
VMs
Databases
32 © Copyright 2011 EMC Corporation. All rights reserved.
Production Site
Outage
Protecting SharePoint Business Continuity – vCenter SRM with RecoverPoint CRR
33 © Copyright 2011 EMC Corporation. All rights reserved.
Test type Shutting down
production VMs Preparing storage
Restarting DR
VMs Total time
Executing recovery plan for a
fully operational farm (under load) 00:33 11:45 3:15 15:33
Testing recovery plan for a fully
operational farm serving user
requests 00:00 11:36 4:32 16:08
Protecting SharePoint Business Continuity – vCenter SRM with RecoverPoint CRR
Proven Solution Test Results
– 13,080 heavy users @ 10% concurrency - Sustained Performance
– 30% reduced cost of BLOB Storage with EMC LUN Compression
– 6 minutes to restore 100GB Content Database
– <16 minutes to perform full end-to-end failover
– 27 minutes to provide local first-time protection with Replication Manager
http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf
Failover Test Results
34 © Copyright 2011 EMC Corporation. All rights reserved.
Key Takeaways
• SharePoint is more than just SQL…
– Leverage EMC Proven solutions and Best Practices for SharePoint storage,
networking and compute design.
– FAST, FAST Cache, VP improve efficiency & performance but require proper planning.
– Use RBS to improve scalability and TCO and in some cases, performance.
• Full farm virtualization has great advantages over physical/hybrid configurations.
– Horizontal scaling is more efficient
– The best FULL farm protection when Integrated with EMC replication.
– Simplifies, accelerates and automates SharePoint DR! (SRM, CE)
• EMC’s SharePoint VSS based replication can significantly accelerate replication
and recovery of SharePoint
– A must for large deployments (TBs)
– Protects content and index
– Fast and simple Item level recovery while integrating with EMC partners (e.g. Kroll)
35 © Copyright 2011 EMC Corporation. All rights reserved.
Additional References
EMC Solutions for SharePoint Portal – http://www.emc.com/sharepoint
Technical Whitepapers
• Virtualized SharePoint 2010 (ESX 4.0, CX4-240)
– http://www.emc.com/collateral/software/white-papers/h8024-virtual-sharepoint-clariion-vsphere-wp.pdf
• Continuous protection for virtual SharePoint 2010 (ESX 4.1, RBS, RP, RM, CX4-120)
– http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf
• SharePoint 2010 BLOB externalization (Hyper-V, Metalogix StoragePoint, VNX5300) – http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf
• SharePoint 2007 Business Continuity (Hyper-V, Cluster Enabler, CX4-120) – http://www.emc.com/collateral/solutions/reference-architecture/h7041-bc-sharepoint-clariion-recoverpoint-
hyperv-ref-arc.pdf
SharePoint Blogs - EMC – http://sharepointintheprivatecloud.wordpress.com
– http://sustainablesharepoint.com
36 © Copyright 2011 EMC Corporation. All rights reserved.
Q&A
37 © Copyright 2011 EMC Corporation. All rights reserved.
THANK YOU