Date post: | 03-Jan-2016 |
Category: |
Documents |
Upload: | thomasina-lee |
View: | 220 times |
Download: | 2 times |
1© Copyright 2012 EMC Corporation. All rights reserved.
EMC PERFORMANCE OPTIMIZATION FOR MICROSOFT FAST SEARCH SERVER 2010 FOR SHAREPOINTEMC Symmetrix VMAX, FAST VP, Microsoft Hyper-V
2© Copyright 2012 EMC Corporation. All rights reserved.
Agenda
Solution overview: Objectives
Why this solution
Solution architecture (physical and virtualized)
Results and findings (physical and virtualized)
Backup and restore results
Summary
3© Copyright 2012 EMC Corporation. All rights reserved.
Solution overview: Objectives Implement Microsoft FAST Search Server 2010 for SharePoint in
a virtualized environment on a Symmetrix VMAX storage system
– Validate and show a FAST Search environment on Physical servers for 4 TB SharePoint content and 5 TB file share document
– Showcase how to configure and deploy FAST Search servers
– Provide guidelines for virtualizing the FAST Search servers
– Validate the benefit of FAST VP for the SharePoint farm environment as well as the FAST Search server environment
– Validate the SharePoint and FAST Search full farm backup, content database restore and FAST Search full farm backup and restore
4© Copyright 2012 EMC Corporation. All rights reserved.
Why this solution Microsoft FAST Search Server 2010 for SharePoint delivers an
exceptional intranet and people search experience and a platform for building custom search-driven applications.
The deployment of the FAST Search Server 2010 for SharePoint needs careful planning and understanding of both SharePoint and FAST Search servers. This solution can provide some guidance to the implementation team.
Virtualization is often called for from many customers, but there is no guideline available. Questions such as “Is it possible?” “How could it be done?” “What kind of performance can I get?” remained unanswered.
This solution addresses the questions above with EMC VMAX as the storage platform and Microsoft Hyper-V as the virtualization platform.
6© Copyright 2012 EMC Corporation. All rights reserved.
FAST Search for SharePoint topology overview (I)
FAST Search Server 2010 for SharePoint provides enterprise search infrastructure for the SharePoint server farm.
The search solution is hosted on two farm topologies:
– SharePoint Server farm hosts the query interfaces and content collection (crawling) interfaces for the search solution.
– FAST Search Server 2010 for SharePoint farm provides back-end indexing and query matching capabilities and administration of these features.
7© Copyright 2012 EMC Corporation. All rights reserved.
FAST Search for SharePoint topology overview (II)
FAST Search for SharePoint farm topology and work flow
8© Copyright 2012 EMC Corporation. All rights reserved.
Design considerations Use minimum servers in a FAST Search farm
Create a redundant Web analyzer to ensure service availability whenever it is possible
Keep storage out of the administrator server for easy recovery
Install FAST on the non-operating system LUN to divert the logging activities and make LUN based replication of the FAST Search farm possible
9© Copyright 2012 EMC Corporation. All rights reserved.
Solution architecture:
Physical FAST Search farm design
10© Copyright 2012 EMC Corporation. All rights reserved.
Solution architecture with physical FAST Search servers (I)
11© Copyright 2012 EMC Corporation. All rights reserved.
Solution architecture with physical FAST Search servers (II)
Physical FAST Search server farm configuration
12© Copyright 2012 EMC Corporation. All rights reserved.
Solution architecture:
Virtualized FAST Search farm design
13© Copyright 2012 EMC Corporation. All rights reserved.
Solution architecture with virtualized FAST Search servers (I)
14© Copyright 2012 EMC Corporation. All rights reserved.
Solution architecture with virtualized FAST Search servers (II)
Virtualized FAST Search server farm configuration
16© Copyright 2012 EMC Corporation. All rights reserved.
Testing profiles User load profile
– Heavy (60 requests/hour)– Required response time
Content profile
Test type Details Required response timeBrowse User browse Less than 3 secondsSearch Unique value search Less than 3 secondsModify Browse and metadata modify Less than 3 seconds
Item ValueContent type doc, docx, pdf, xls, xlsx, ppt, pptx, txtAverage content size 1.65 MBSharePoint - Total data 4 TB SharePoint – Document size range 20 KB – 50 MBSharePoint – Number of documents 2,000,000 SharePoint – Total site count 220 SharePoint content database size (each) Up to 200 GB
SharePoint – Total site collections 20 SharePoint – Sites for each site collection 11 SharePoint – Total user count 22,000 SharePoint -- User concurrency 10% File share – Total data 5 TB File share – Number of documents 3,000,000
17© Copyright 2012 EMC Corporation. All rights reserved.
Key results (I)
Environment Available CPUsDocument processors
Full crawl rate ( items/minute)
SharePoint content
File share
Physical 60 42 1513 1667
Virtualized 20 24 770 978
Server typeAverage read size (KB)
Average write size (KB)
Average read/write
Total IOPS
SQL server 26 24 218:1 1434
FAST index 275 605 3:1485
FAST servers 13 26 1:66
Environment
Profile ( browse/ search/ modify)
Average response time (second)
Queries per second
FAST IOPS total
Browse Search Modify
Physical 80/10/10 0.94 0.60 2.43 7.1
550
Virtualized 80/10/10 0.95 0.47 2.83 8.5
18© Copyright 2012 EMC Corporation. All rights reserved.
Key results (II) The maximum total disk IOPS is around 3,500 on the back end for full
crawls and around 1,900 for the query testing.
Crawl rate is 150 GB/hour in the physical environment, and 80 GB/hour in the virtualized environment with less hardware resources.
Search response time is less than 1 second in both physical and virtualized environments supporting 22,000 users with 10% concurrent access.
19© Copyright 2012 EMC Corporation. All rights reserved.
Key results (III)
With Replication Manager-engaged TimerFinder snapshot backup: – The whole SharePoint farm, with a total of 9 TB data, can be
backed up within 15 minutes. – The FAST Search farm, with a total of 5 TB FAST Search
volumes, can be backed up within 8 minutes.
With Replication Manager-engaged TimerFinder snapshot restore:– The SharePoint content database (4 TB) and file share data
(5 TB) can be restored and recovered within 1 hour and 30 minutes.
– The FAST Search farm, with a total of 5 TB FAST Search volumes, can be restored within 13 minutes.
Replication Manager-engaged TimeFinder/Snap backup and restore
20© Copyright 2012 EMC Corporation. All rights reserved.
Key results (IV)FAST VP With FAST VP enabled, the environment can better handle peak
I/O request.
The search performance improved about 20% after enabling FAST VP on Symmetrix VMAX storage.
QPS result of profile 80/10/10 (percentage of browse/search/modify)
FAST VP used? (Yes /No) Queries per second (QPS)
No 7.1
Yes 8.5
Before FAST VP was enabled After FAST VP was enabled
100% FAST volumes on FC virtual pool
1.3% on Flash virtual pool
94.6% on FC virtual pool
4.3% on SATA virtual pool
22© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for SharePoint content in the physical FAST Search farm (I)
Full crawl performance of the SharePoint content
IOPS and CPU usage of the SharePoint full crawl
Duration (hh:mm:ss)
Items crawled Crawl rate (items /min)
Crawl rate (MB/min)
Crawl type
24:15:00 2,190,954 1,513 2,111 Full
Item FAST Search servers SharePoint servers
Average total IOPS 485 1,434
Maximum total IOPS 1,809 4,749
Average CPU usage 41% 26%
Available CPUs in the farm 60 N/A
Document processors 42 N/A
FAST Search index volume I/O characteristics and read/write ratio
Server role Average IOPSIOPS (maximum)
Read I/O size
Write I/O size
Read:Write
Primary 345 1,809 275 KB 605 KB 3:1
Secondary 47 334 45 KB 664 KB 1:50
23© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for SharePoint content in the physical FAST Search farm (II)
SharePoint content database full crawl rate
• The crawl rate of smaller documents was higher with the peak reaching 8,000 items per minute.
• The crawl rate of larger documents was lower, which was 1,516 items per minute or 2111 MB per minute on average.
24© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for SharePoint content in the physical FAST Search farm (III)
Back-end storage Symmetrix VMAX performance
• Disks and thin pools performed well• The disk utilization of SharePoint and FAST Search
volumes was less than 50 percent • The total IOPS on the back end was 2,500 to 4,500
Item Average Maximum
IOPS 3,340 4,730
Throughput (MB)
210 275
Disk group utilization
Percentage
Content DB 5%–35%
FAST index volumes 10%–47%
25© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for file share in the physical FAST Search farm (I)
Full crawl performance of the file share content
IOPS and CPU usage of the file share full crawl
Duration (hh:mm:ss)
Items crawled Crawl rate (items/min)
Crawl rate (MB/min)
Crawl type
35:15:00 3,124,223 1,668 2,602 Full
Item FAST Search servers SharePoint servers
Average total IOPS 208 339
Maximum total IOPS 1,706 930
Average CPU usage 45% 34%
Available CPUs in the farm 60 N/A
Document processors 42 N/A
FAST Search index volume IO characteristics and read/write ratio
Server roleAverage IOPS
IOPS (maximum)
Read I/O size
Write I/O size
Read:Write
Primary 284 1,706 313 KB 617 KB 5:2
Secondary 45 186 18 KB 756 KB 1:25
26© Copyright 2012 EMC Corporation. All rights reserved.
• The crawl rate was higher than the content database
• The crawl rate was largely related to the size of the files peak while processing the smaller files
Test result: Full crawl for file share in the physical FAST Search farm (II)
File share full crawl rate in the physical environment
27© Copyright 2012 EMC Corporation. All rights reserved.
• Disks and thin pools performed well
• Disk pools utilization was low
• Total IOPS on the back end was around 1400-2800
Back-end storage Symmetrix VMAX performance
Test result: Full crawl for file share in the physical FAST Search farm (III)
Item Average Maximum
IOPS 2,200 2,880
Throughput (MB)
234 322
Disk group utilization
Percentage
Content DB 5%–25%
FAST index volumes 20%–55%
28© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Query performance on the physical FAST Search farm (I)
VSTS (browse/search/modify)testing
Profile (percentage of browse/search/modify)
Requests per second
Average response time (second)
Queries per second
Browse Search Modify
80/10/10 35.7 0.94 0.60 2.43 7.1
70/20/10 36.5 0.93 0.63 2.14 9.5
50/30/20 38.0 0.90 0.83 2.26 16.7
29© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Query performance on the physical FAST Search farm (II)
Back-end storage performance with VSTS testing
Symmetrix VMAX back end IOPS during 80/10/10 (percentages for browse/search/modify) profile
Maximum IOPS
Total SharePoint and FAST Search servers 1,900
FAST Search servers only 550
31© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for SharePoint content in the virtualized FAST Search farm (I)
Full crawl performance of the SharePoint content
IOPS and CPU usage of the SharePoint content full crawl
Duration (hh:mm:ss)
Items crawled Crawl rate (items/min)
Crawl rate (MB/min)
Crawl type
47:47:00 2,190,954 770 1,395 Full
Item FAST Search servers SharePoint servers
Average total IOPS 675 900
Maximum total IOPS 6,592 4,257
Average CPU usage 74% 45%
Available CPUs in the farm 20 N/A
Document processors 24 N/A
FAST Search index volume I/O characteristics and read/write ratio
Server roleAverage IOPS
IOPS (maximum)
Read I/O size
Write I/O size
Read:Write
Primary 483 6,592 184 KB 345 KB 3:1
Secondary 103 13,346 42 KB 582 KB 1:5
32© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for SharePoint content in the virtualized FAST Search farm (II)
Full crawl rate of the SharePoint content
• The crawl rate of smaller documents was higher with the peak reaching 12,500 items per minute.
• The crawl rate of larger documents was lower, which was 770 items per minutes or 1395 MB per minutes on average.
33© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for SharePoint content in the virtualized FAST Search farm(III)
• Disks and thin pools performed well
• Disk pools utilization low
• Total IOPS on the back end was around 2,800-4,300
Back-end storage Symmetrix VMAX performance
Item Average Maximum
IOPS 3,500 4,300
Throughput (MB)
217 300
Disk group utilization
Percentage
Content database 20%–30%
FAST index volumes
30%–50%
34© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for file share in the virtualized FAST Search farm (I)
Full crawl performance of the file share content
IOPS and CPU usage of the file share full crawl
Duration (hh:mm:ss)
Items crawled Crawl rate (items/min)
Crawl rate (MB/min)
Crawl type
60:14:32 3,124,223 978 1383 Full
Item FAST Search servers SharePoint servers
Average Total IOPS 384 193
Maximum Total IOPS 7,869 1,263
Average CPU usage 83% 15%
Available CPUs in Farm 20 N/A
Document Processors 24 N/A
FAST Search index volume I/O characteristics and read/write ratio
Server roleAverage IOPS
IOPS (maximum)
Read I/O size
Write I/O size Read:Write
Primary 463 7,869 186 KB 373 KB 3:1
Secondary 29 8,704 36 KB 560 KB 1:2
35© Copyright 2012 EMC Corporation. All rights reserved.
• The crawl rate was faster than the content database.
• The crawl rate was largely related to the size of the files peak while processing the smaller files; same pattern with the physical environment.
Full crawl rate of the file share content
Test result: Full crawl for file share in the virtualized FAST Search farm (II)
36© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Full crawl for file share in the virtualized FAST Search farm(III)
• The total IOPS on the back end was between 2,500 and 4,300.
• The disks and thin pools performed well. Disk utilization of SharePoint and FAST Search volumes was less than 50 percent.
Back-end storage Symmetrix VMAX performance
Item Average Maximum
IOPS 3,500 4,300
Throughput (MB)
217 300
Disk group utilization
Percentage
Content database 20%–30%
FAST index volumes
30%–50%
37© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Query performance on the virtualized FAST Search farm (I)
VSTS testing
Profile (percentage of browse/search/modify)
Requests per second
Average response time (second)
Queries per second
Browse Search Modify
80/10/10 38.2 0.95 0.47 2.83 8.5
70/20/10 41.6 0.90 0.55 2.50 14.4
50/30/20 38.5 0.88 0.67 2.08 12.9
38© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Query performance on the virtualized FAST Search farm (II)
Back-end storage performance with VSTS testing
Symmetrix VMAX back end IOPS during 80/10/10 (browse/search/modify) profile
Maximum IOPS
Total SharePoint and FAST Search servers 3,400
FAST Search servers only 850
39© Copyright 2012 EMC Corporation. All rights reserved.
FAST VP test results• FAST VP policy:
– Flash: 20 percent
– Fibre Channel: 30 percent
– SATA: 50 percent
• Before FAST VP was enabled: 100 percent FAST Search volumes were on FC virtual pool.
• After FAST VP was enabled: About 1.3 percent of data was promoted to the Flash tier; about 4.3 percent of data was demoted to the SATA tier; and 94.6 percent of data resided on the FC tier.
In the test above, the search performance improved about 20 percent with FAST VP enabled.
40© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Replication Manager-engaged FAST farm backup and restore (I)RM-engaged backup/restore summary• Backup:
– A snapshot for the whole SharePoint farm took only 11 minutes to complete.
– A snapshot for file share completed in less than 2 minutes.
– A snapshot for the FAST Search farm took about 8 minutes across three servers.
• Restore:
– Restoring a single SharePoint database took only a few minutes to process. Restoring all 20 SharePoint database across two SQL Server database servers took about 1.5 hours, most of which was spent recovering the databases (log replay).
– Restoring file share took less than 2 minutes to finish.
– Restoring the FAST Search farm took less than 13 minutes to finish.
41© Copyright 2012 EMC Corporation. All rights reserved.
Test result: Replication Manager-engaged FAST farm backup and restore (II)Replication Manager-engaged TimeFinder/Snap backup
Replication Manager-engaged TimeFinder/Snap restore
*Including database recovery time.
Item Total sizeBackup duration (hh:mm:ss)
SharePoint farm backup 4 TB 0:11:33
File share backup 5 TB 0:01:53
FAST Search farm backup
5 TB0:07:50
Item Total sizeRestore duration (hh:mm:ss)
SharePoint content database restore*
4 TB1:24:33
File share restore 5 TB 0:01:53
FAST Search farm restore
5 TB0:12:49
42© Copyright 2012 EMC Corporation. All rights reserved.
Configuration tipsThe following settings are recommended by Microsoft for better crawl performance.
• SharePoint crawler registry changes:
– HKLM\SOFTWARE\Microsoft\Office Server\14.0\Search\Global\Gathering Manager• FilterProcessMemoryQuota
– Default 100 MB– Changed to 200 MB
• DedicatedFilterProcessMemoryQuota– Default 100 MB– Changed to 200 MB
• FolderHighPriority– Default 50 – Changed to 500
• Turn off TCP/IP Chimney: http://support.microsoft.com/kb/2570111
• Apply hotfix KB976462
• Document processors could be adjusted in the run time
43© Copyright 2012 EMC Corporation. All rights reserved.
Summary (I)• SharePoint farm design
– The SharePoint farm used three Hyper-V servers supporting 12 virtual machines, which formed the virtualized SharePoint farm.
– The SharePoint farm contained 9 TB content source (4 TB content database and 5TB file share).
– The SharePoint farm supported 22,000 users to operate with mixed profile (browse, search, and modify) with 10 percent concurrent access. The average search response time was less than 1 second.
• FAST Search farm design
– The physical FAST Search farm used five physical servers.
– The virtualized FAST Search farm used two physical hosts with Hyper-V enabled.
• FAST Search performance in a virtual environment
– A full crawl against a 5 TB file share can be completed in 60 hours and 15 minutes in the virtualized FAST Search servers (35 hours and 15 minutes in the physical FAST Search servers).
– A full crawl against 4 TB of SharePoint content can be completed in 47 hours and 15 minutes in the virtualized FAST Search servers (24 hours and 15 minutes in the physical FAST Search servers).
44© Copyright 2012 EMC Corporation. All rights reserved.
Summary (II)• Consolidation ratio in a virtualized environment
– The migration of FAST Search servers to a virtualized environment reduced the number of physical host servers from five to two (60 percent reduction). This also leveraged all the management advantages of a virtualized environment while achieving comparable query performance of a physical environment and acceptable crawl performance.
• I/O characteristics of FAST Search components
– The read and write I/O size were around 300 KB and 600 KB on the primary index volume of the FAST Search server.
– The read and write I/O size were around 40 KB and 700 KB on the secondary index volume of the FAST Search server.
• Enabling FAST VP improved the query performance of FAST Search by 20 percent
45© Copyright 2012 EMC Corporation. All rights reserved.
Summary (III)• Crawl rate
– A full crawl against a 5 TB file share completed in 35 hours and 15 minutes in the physical FAST Search servers, or 60 hours and 15 minutes in the virtualized FAST Search servers.
– A full crawl against 4 TB of SharePoint content completed in 24 hours and 15 minutes in the physical FAST Search servers or in 47 hours and 15 minutes in the virtualized FAST Search servers.
• Search Rate
– Mixed workload with 30 percent query achieved 16.7 queries per second (QPS) in the physical FAST Search farm.
– 14.4 QPS were processed in the virtualized FAST Search farm while supporting 22,000 users with 10 percent concurrent access.
– Pure search test achieved 205 QPS while keeping search the mean query latency response time less than 0.072 seconds.
• Backup and Restore
– Replication Manager ensures that the environment is protected. The SharePoint content and FAST Search farm were replicated and recovered with minimal impact on the users.