©2012 Quest Software, Inc. All rights reserved..
Chris McNulty
SharePoint Strategic Product Manager
April 2012
SharePoint Saturday Twin Cities
Optimizing and Accelerating your SharePoint Farm
©2012 Quest Software, Inc. All rights reserved. Confidential.
2
©2012 Quest Software, Inc. All rights reserved..
2
AmericasEMEAAPJa
Sales/MrkgR&DSupport
178 CountriesAll VerticalsGlobal 200SMB
Database, Monitoring, Data Protection, User Workspace/Virtualization, Windows (SharePoint, AD, Messaging), Identity Mgmt
60 Offices, 3 HQs
3600+ Employees
100,000+ Customers
Quest Market Presence
Multiple Business Lines
©2012 Quest Software, Inc. All rights reserved. Confidential.
3
©2012 Quest Software, Inc. All rights reserved..
3
4
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Chris McNulty• SharePoint Strategic Product
Manager at Quest Software
• 10+ years with SharePoint
• 20 years consulting (led KMA SharePoint practice) and financial services technology (Santander, John Hancock/Manulife, GMO, State Street)
• MBA in Inv Mgmt from Boston College
• Write and speak often on Microsoft IW technologies (blogs & books)
• MCSE MCTS MSA MVTSP MCC
• Hiking, cooking, playing guitar, colonial history, photography
• My family: Hayley, three kids (17, 8, 5) and my dog Stan
5
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Pictures
• [taken out to save space!]
In the spirit of…
7
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Hometown News• National: Newark NJ Mayor in Rescue (MSNBC)
• Newark, N.J., Mayor Cory Booker suffered second-degree burns and smoke inhalation as he helped rescue one of his next-door neighbors from a burning building Thursday. “There was a time when I didn’t think we’d make it out of there,” Booker told media at a news conference Friday morning, describing the rescue as his "proverbial 'come to Jesus' moment."
• Local: (Suffolk Times, NY) : Cops: 89-year-old woman crashes car into pole in Cutchogue • An 89-year-old woman crashed her car into a utility pole on Main
Road in Cutchogue Thursday afternoon, Southold Town Police officials said. At about 3:30 p.m., Margaret Spicijaric of Southold was traveling alone in a silver 2002 Honda heading eastbound before crashing into a utility pole, officials said.
• Weather (Minneapolis area, from weather.com)• Sun and clouds mixed with a slight chance of thunderstorms
during the afternoon. Warm. High 73F. Winds S at 5 to 10 mph. Chance of rain 30%.
• Overnight lows 56, chance of rain or thundershowers 90%
8
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Agenda
• Architecture, Design & Planning
• Support• Monitoring and Operations
• Optimization
• Patching
• SQL Maintenance
• Metrics
• Best Practices
9
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Presentation Governance
• Out Of Scope• Deep Dives (e.g. PowerShell, BI, Upgrade, SQL DBA)
• Development
• Customization
• Power User (e.g. Library Customization, Designer Workflows, etc.)
• Rules• Move fast, PowerPoint is shared –
http://slidesha.re/GztaOY
• Questions – time permitting during session
• Any time after session – email etc. - @cmcnulty2000
Architecture and Design
11
©2012 Quest Software, Inc. All rights reserved..
Server Farm – Web Front End
• Typical Roles:• http services
• Search query
• Scaling• Add servers to load
balanced cluster
• Performance Optimization
• RAM
• Easily virtualized
12
©2012 Quest Software, Inc. All rights reserved..
Server Farm - Application Server
• Typical Roles:
• Search index/crawl
• Excel calculation
• User profiles
• Managed Metadata
• Scaling
• Add search servers and partitions
• Move shared services to dedicated servers
• Performance Optimization
• CPU
13
©2012 Quest Software, Inc. All rights reserved..
Server Farm - Database
• Typical Roles:• Data storage
• SQL Reporting
• Scaling• Add storage capacity
• Performance Optimization
• Disk I/O
14
©2012 Quest Software, Inc. All rights reserved..
Sizing - Single Server
• Typical Roles:• Small teams
• Small document pools
• Considerations• Performance & fault
tolerance less of a concern
• SQL & Web on same system
• Search not a core function
15
©2012 Quest Software, Inc. All rights reserved..
Sizing - Medium Farm
• Typical Roles:• 100-10,000 users
• 10,000 – 1MM documents
• Scenarios• Enterprise portal
• Large scale collaboration
• Broader applications platform
• Larger external search pool
• Mix and match internal external front end servers on common content databases
16
©2012 Quest Software, Inc. All rights reserved..
Sizing - Large Farm
• Typical Roles:
• Large distributed enterprise users (10000+)
• Large pools of documents (>1MM)
17
©2012 Quest Software, Inc. All rights reserved..
Sizing – No Servers – Office 365Office 365 Enterprise Plans
E1 E2 E3 E4
SharePoint Online
√ √ √ √
Office Web Apps
√ √ √
Local Copy of Office Professional 2010 Plus
√ √
Forms Services, Vision Services, Access Services
√ √
Monthly cost per user (USD)
$10 $16 $24 $27
• Constraints/Unavailable• Custom, non sandbox solutions
• Power Pivot
• SQL Server Reporting Service Integration
• Business Connectivity Services (OK for web services- based remote data in O365 BCS.)
• FAST Search Server Integration
• Web Analytics
• Site collections greater than 100GB
18
©2012 Quest Software, Inc. All rights reserved..
Logical Components
• High capacity!
• Maximums
• 250,000 sites per site collection
• 5,000 site collections per content DB
• 200GB max content DB (single site collection)
• >200GB post SP1
• 300 Content DBs per web application
• 30MM documents/library
• 2GB document size
• 2011 News• 14TB Demo
19
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Disk SizingContent Search
Initial Content Size XXX GB External Crawl Size YYY GB
Initial User Pool U
User Collab Size .25GB
n YR Growth Rate – Archive Rate
G%
End Content Size XXX (1+G)n = ECS End Search Size YYY (1+G)n = ESS
End User Collab Size .25 * U * (1+G)n = EUCS
Content DBs ECS + EUCS
Search DBs .05 * (ECS + EUCS + ESS)
Search Index Files .05 * (ECS + EUCS + ESS)
• Inputs: Size of SharePoint content and non-SharePoint content included in search
• For DBs, don’t forget transaction logs, disk dumps (if used for backup) which can add 1-3X.
• In SAN or virtual environments, not all disk need be provisioned early
Content will grow; searching remote stores saves disk but isn’t free
20
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Memory Sizing
• Web Front End (WFE)• 8GB minimum
• 12GB larger
• 16GB max
• Application• 8-12GB
• SQL• 8-24GB
• HP Sizing Tool• http://
h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html
• Don’t forget about the swap files (1-2X size of RAM)
21
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Psst…about SQL…
• Sidebar on SQL Disk optimization• RAID 1: Mirroring (Wastes disk)
• RAID 5: Parity Bit (write performance hit)
• RAID 10 Stripes across mirrors (costly)
• Physical location of data, log, temp and/or backup files. If virtualization or SAN technology doesn’t isolate the disks, not much performance gain,
• Performance optimization/fault tolerance by:• RAID1 on boot disks
• RAID5 on data disks
• RAID10 on log disks
• No RAID, or RAID 5 on backup disks
• RBS reduces size (and count) of content databases but doesn’t reduce size of total storage
• IOPS, IOPS, IOPS!
22
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Database sizing
• Automatic database growth:• 50-100MB clumps – not by percentage.
• Using a small size leads to more frequent, but smoother, steady state growth.
• Presize tempdb to about 20% the size of the single largest content database.
@cmcnulty2000
MonitoringMonitoring and Operations
24
©2012 Quest Software, Inc. All rights reserved..
Proactive Issue Resolution
Developer Dashboard• Empower developers and
users
Integrated Health Analyzer• Runs when necessary
• Alerts anomalies
• Fixes when it can
Web Analytics• User usage
• Resource usage
25
©2012 Quest Software, Inc. All rights reserved..
Logging, Monitoring, and Alerts
• Unified Logging
• Out-of-the-box reports
• Richer Web Analytics
• Open Schema
• SCOM Integration
• PLUS• Developer Dashboard
• Health Analyzer
ULS Logs
Windows Events
Page requests
Feature Logging
Health data
Logging DB
27
©2012 Quest Software, Inc. All rights reserved..
Developer Dashboard Improve customized solutions with the Developer Dashboard
SQL Queries Performance
Memory Usage
Data-Request Trace
28
©2012 Quest Software, Inc. All rights reserved..
Monitoring – WSS_Logging
• Query Database Views Directly
• Requires Timer Jobs Enabled• Diagnostic Data Provider: Trace Log
• Diagnostic Data Provider: Event Log
• ULS Configuration Matters
• Database will GROW!
• Aggregates from ALL Servers
• Sample: • SELECT * FROM [WSS_Logging].
[dbo].[ULSTraceLog] WHERE CorrelationID = '04377DAE-C2FD-4DBE-A57E-101B3005059E'
30
©2012 Quest Software, Inc. All rights reserved..
Development Support – Three RegionsDevelopment
• often internal to developers
• problem reproduction that require advanced inspection tools (e.g. Visual Studio) are done here
• permissions can be looser, may have multiple environments for multiple developers
• sensitive data from production cannot be copied here without masking or customer signoff
• changes here can be deployed ad hoc
Staging/Test
• no Visual Studio, no MS Office
• match/mirror production as closely as possible; match hardware/system performance as closely as practical
• security permissions match production
• any sensitive data copied here stays under production-grade controls
• test accounts should be created in a separate OU if possible
• changes here can only be delivered and deployed from source control and according to production release methods
Production
• optimized hardware configurations
• highly secure• no use of user rotating
password accounts as service accounts
• changes here can only be delivered and deployed from source control and according to production release methods
31
©2012 Quest Software, Inc. All rights reserved..
demoMonitoringDeveloper DashboardHealth AnalyzerLogging and Reporting
@cmcnulty2000
OptimizationOptimization
33
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Optimization
• IIS Bit Rate Throttling (Video)• Part of IIS Media Services 2.0 or greater
• Download http://www.iis.net/download/BitRateThrottling
• Configured through IIS Manager
• Large File Upload Support:• Central Admin: 600MB Upload Size
• IIS7 Connection Timeout 65536 seconds
• Web app web.config = change line to • <httpRuntime maxRequestLength="2097151" executionTimeout="999999" />
• in 14hive\template\layouts\web.config, change above line to same
• Finally, as per http://support.microsoft.com/kb/944981/en-us add section to web.config to allow 600MB content length (it’s a new requirements in IIS7).• <system.webServer>
• <security>
• <requestFiltering>
• <requestLimits maxAllowedContentLength="629145600"/>
• </requestFiltering>
• </security>
34
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Optimization
• Disk-based BLOB Caching• Local store for audio/video, PDF other frequent read only files
• Edit in Web.config (C:\Inetpub\wwwroot\wss\virtualdirectories\...)
• <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />
• Location = Local Disk Location
• maxSize = GB
• Enabled = true
• Different from RBS/EBS!
• For publishing sites
35
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Patching – Notes
• Test before installation!!!
• Cumulative Updates every two months
• Service Pack every 6-18 months
• Service Pack 1 REQUIRES at least the June 2011 CU
• December 2011 CU • iOS 5 Mobile Safari support for Performance Point
• Fixed Administrator updates to user profile pictures
• February 2012 CU most current
• Check my blog for latest: http://www.chrismcnulty.net/blog/Lists/Categories/Category.aspx?CategoryId=5&Name=Version-Build Numbers
36
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
SQL Maintenance
• Backups• Local Disk – easy but storage intensive
• Agents – remote, requires extra software
• RBS Maintenance• BLOB Orphans
• Log Sizing• Full logged (default) generates huge t-logs
• Simple doesn’t but prevents point in time restore
• Maintenance Plans
37
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Metrics and Results
• IOPS <400• Improve disk I/O or reduce content database sizes
• Processor/CPU utilization >60%• You may need to add servers or redistribute services to reduce the load.
• Available memory <50%• Again, add RAM.
• Disk average reads/writes >15ms. • ~66.67 IOPS Common on virtualized SQL servers. That’s slow, and predicts
reduced performance even for smaller content databases on a SQL server. Use physical disk or move the volume to dedicated storage.
• Sustained network bandwidth higher than 25%. Network latency output queue >0• Network can’t keep up with users or the server. Add physical switched Ethernet
ports, adding servers to divide the load.
• Or move a virtualized guest server to a new host.
38
©2012 Quest Software, Inc. All rights reserved..
demoIOMeterPerformance Monitor
Best Practices
40
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Troubleshooting – Top Support Questions
• Users Receive “Cannot Connect to Configuration Database” Web Page• SharePoint farm account is locked out
• No one can upload anything but site is up• Database disk volume is full – check transaction logs, backups
• In virtualized environment, host file systems may be full
• I can’t find a document I think I should see; Someone can’t see a file I just uploaded• Security and permission variations
• Document “movement” (a/k/a ECM) try search by name or Document ID. Check ECM logs/audits
• Confirm permissions, and make sure document is checked in (Required properties may be missing)
41
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Troubleshooting – More Support Questions
• Repeated requests to re-enter Windows credentials• Add to Local intranet zone, add site, custom level, automatic login with
current user name and password (it’s the last thing in the item list)
• OR Trusted sites
• My workflow didn’t start• Recycle timer service
• “FixSharePoint.exe” = IISReset & Timer Service Recycle
• I’m not seeing the right search results• Confirm that crawls are running and complete by checking crawl logs;
restart a full crawl if crawls finish OK
• I need a file back that I deleted• Recycle Bin Recovery
• Use Backup & Restore
42
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Miscellaneous Tips and Tricks I• Bad web parts on page
• ?Contents=1 on end of URL
• HyperV & DiskPart• From the run menu type "diskpart.exe" to enter the command
line utility to resize disk partitions in Windows Server 2003.
• The command list volume will show you all the available volumes.
• Select your volume as shown below. select volume 1 corresponds to the "D" volume.
• Extend the volume with the extend command; the partition will be immediately extended under the Disk Management snap in.
• User Profile Sync• DirSync permissions in AD are essential (Replicating Directory
Changes )
• PSCONFIG for a stuck wizard• psconfig.exe -cmd upgrade -inplace b2b -wait
43
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Miscellaneous Tips and Tricks II• How do I make a bigger template?
• stsadm -o setproperty -propertyname max-template-document-size -propertyvalue size_in_bytes
• Auditing• In Central Admin | Security, configure Information
Management Policy (/_admin/Policyfeatures.aspx). Make sure Auditing is Available
• At the Site Collection Audit Settings screen (in Site Collection Administration) select the events you want to audit
• Site Setting | Site Collection Administration | Audit Log Reports
• Service Account Management in Central Admin• Security | Managed Service Accounts
• When all else fails – web.config!!!!
48
©2012 Quest Software, Inc. All rights reserved..
•Questions
•Contact Me
•SharePint
•Prizes
49
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
More information
• SharePoint architecture design patterns in Chris’ e-book entitled SharePoint 2010 Consultant’s Handbook – A Practical Field Guide • Get your free copy here http://www.quest.com/get-chris-book
50
©2012 Quest Software, Inc. All rights reserved..
@cmcnulty2000
Thank you…
• Contact• Email [email protected]
• Blog http://www.chrismcnulty.net/blog
• Also http://www.sharepointforall.com
• Twitter: @cmcnulty2000
• LinkedIn:http://www.linkedin.com/in/cmcnulty
• Upcoming:• April 2012 – The Experts Conference San Diego
• May 2012 – New England SPUG
• June 2012 – SPS Silicon Valley, New Hampshire SPUG
• July 2012 – SPS Toronto, SharePoint Saturday NYC
@cmcnulty2000
©2012 Quest Software, Inc. All rights reserved..