Post on 23-Dec-2015
transcript
Consolidation of a Performance Sensitive Application: Virtualizing Electronic Sports League's Gaming Infrastructure
Breakout Session PO3435
Bob Albers Sreeram SammetaI/O Usage Architect, Intel Senior Systems Engineer, Intel2008-08-26
Server Applications:
• Can We Virtualize* Everything?
• New Technologies Allow Us To Virtualize* More
• Question The Assumptions
• ROI Delivered
Agenda
*In this context “virtualize” means to use a VMM or hypervisor
Can We Virtualize Everything?Can We Virtualize Everything?
Perception: “Non-virtualize-able” Applications
Mission Critical
Transaction Latency Sensitive
CPU Intensive: Single Thread vs. Multi-Thread
Memory Intensive: Size/Throughput/Latency
I/O Intensive: Disk/Network; Throughput/Latency
Perception: Virtualization impacts all of the above
Therefore: These types of applications “can’t be virtualized” because the tradeoffs are too severe
Perception or Reality?
Electronic Sports League
Largest online gaming community in Europe
>844,000 active users*
Mission Critical Game Servers
Transaction Latency Sensitive
Single Thread CPU Intensive
Can We Virtualize Everything?Can We Virtualize Everything?
“Can’t Be Virtualized” but …
ESL motivated to try: sprawl, cost, power, OpEx
Internet
Game Servers
…
Firewall
* Source: ESL web site 2008-08-12
ESL Proof of Concept (PoC) Overview
Hypothesis: virtualization of gaming servers may be possible if …
We use the latest technologies
Intel® Xeon® Processor 7400-based servers with Intel VMDq NICs
VMware ESX* 3.5U1 & NetQueue
Test network latency impact of virtualization in the Intel lab
Private testing @ ESL lab and public testing on the Internet with real ESL members
New Technologies Allow Us To Virtualize MoreNew Technologies Allow Us To Virtualize More
Prove it!
ESL Network
vSwitch1 vSw0
VM1
VMn
Svc Cons
10 GbE VMDq 1 GbE
Firewall
InternetInternet
Force 10 S50N
PoC Server Hardware
Intel® Xeon® Processor 7400 Series (Dunnington)
HW and SW compatible with existing Quad-core 7300 based platforms
Performance boost from 6-core with new 16 MB L3 cache
Energy efficient boost from 45nm high-k process technology
Enhanced hardware assist features for virtualization
Available September ’08
32 slots256 GB max
32 GB as tested
Configurable PCI Express*
ESB2 I/O Bridge
4 sockets with 6-cores each
7400 Series4x1066 MHz
FBDFBDFBDFBDFBD
FBDFBDFBDFBDFBD
FBDFBDFBDFBDFBDFBDFBDFBDFBDFBD
Intel 7300
New Technologies Allow Us To Virtualize MoreNew Technologies Allow Us To Virtualize More
Importance of I/O Tuned For Virtualization
VMM overhead
Switching load
Interrupt bottleneck
VM1 VM2 VMn
Virtual NIC Virtual NIC Virtual NIC…
VirtualizationHypervisor
NIC
Source: Intel. Throughput measures receive side (Rx) I/O performance of 10GbE LAN.
LAN
Th
rou
gh
pu
t (G
B)
0.0
2.0
4.0
6.0
10.0
8.0
4.0
Result: NIC performance can be up to ~60% under utilized
Unused I/O
capacity
w/o VMDq
Source: Intel.
New Technologies Allow Us To Virtualize MoreNew Technologies Allow Us To Virtualize More
Optimizing I/O For Virtualization
VM1 VM2 VMn…
VMware with NetQueue
NIC with VMDq
LAN
Virtual NIC Virtual NIC Virtual NIC
Th
rou
gh
pu
t (G
B)
0.0
2.0
4.0
6.0
10.0
w/o VMDq
8.0
4.0
Source: Intel.
9.59.2
• 2x throughput• Near native 10GbE
w/ VMDq w/ VMDq Jumbo Frames
Tests measure Wire Speed Receive (Rx) Side Performance With VMDq on Intel® 82598 10 Gigabit Ethernet Controller
New Technologies Allow Us To Virtualize MoreNew Technologies Allow Us To Virtualize More
VMDq & NetQueue
Optimize switching
Load balance interrupts
PoC Software
VMware ESX* 3.5U1 Virtual Center 2.5 VMDq & NetQueue
enabled (16 queues on one 10G port)
1 vCPU per VM 2GB memory per VM
Windows Server* 2003 32-bit
CounterStrike* 1.6
New Technologies Allow Us To Virtualize MoreNew Technologies Allow Us To Virtualize More
Intel® Xeon® Processor 7400 Series based Server
VMware® ESX 3.5 U1
Counter Strike 1.6
Windows Server 2003
32 bit
VM1
Counter Strike 1.6
Windows Server 2003
32 bit
VMn
Enough theory…
let’s move on to engineering & testing!
Key Performance Metric: In-Game Transaction Latency
In-game transaction latency = round-trip network latency + game server processing time
= >20mS typical or 10mS best case on Internet
=~ 5mSec best case for LAN connected players
In -G a m e L a te n c y (m S e c )
0
5
10
15
20
25
B e s t L A NB e s t In te rn e tT yp . In te rn e t
Question The AssumptionsQuestion The Assumptions
How much does virtualization increase round-trip Network Latency?
Source: ESL
Source: ESL observations. Actual performance may vary.
40-200 bytes UDP
2000 bytes UDP
Player Game Server
Round-Trip Network Latency Test
Virtualization increases latency; ~double
VMDq latency is near-native @ 0.13mS for 64-byte packets
Negligible since In-Game Transaction Latency is 5mS best case
Question The AssumptionsQuestion The Assumptions
N e tp e rf 2 .4 .4 U D P Late n cy te s t w ith 8 p ara lle l s tre am s
0 .0 0
0 .0 5
0 .1 0
0 .1 5
0 .2 0
0 .2 5
0 .3 0
6 4 2 5 6 1 0 2 4
P ac k e t S ize (b y te s)
Avg
. lat
ency
(ms)
N a ti ve
V M D q
N oV M D q
Network Latency w VMDq is near Native!
Negligible impact to In-Game Latency!
Source: Intel Lab. Performance measured using the netperf benchmark running on Intel® Xeon® processors 7300 (2.93 GHz). Actual performance may vary.
Source: Intel lab
ESL Game Testing
Private ESL & Public Internet testing revealed no impact on In-Game Transaction Latency!
ROI DeliveredROI Delivered "Playing on virtualized gameservers running on Intel and VMWare
technologies gives professional gamers no disadvantages compared
with playing on a non virtualized server. Everything ran smoothly and I
did not notice anything unusual. A perfect setup for professional
gaming."
—Navid Javadi aka mousesports|Kapio
Source: ESL Lab. Performance Source: ESL Lab. Performance measured using esxtop & power meter measured using esxtop & power meter with ESL reference s/w stack running with ESL reference s/w stack running on Intel® Xeon® processor 7400 on Intel® Xeon® processor 7400 Series based platform. Power Series based platform. Power measurement is wall power for the measurement is wall power for the entire server as tested including 8 SAS entire server as tested including 8 SAS drives. Actual performance may vary.drives. Actual performance may vary.
ESL virtual game servers on Intel® Xeon® Processor 7400 Series based platform
020406080
100
24 32 36 40
# of VMs
CPU
(%)
620640660680700720740
Pow
er (W
atts)
Source: ESL lab
ESL Game Testing Results
ROI DeliveredROI Delivered
"The new Six-Core Intel® Xeon® 7400 series processors
(“Dunnington”) were completely overwhelming in all terms. The Intel Xeon MP servers with Intel VMDq technology enable us to efficiently run our servers with reduced costs and without any
negative impacts."
—Bjoern Metzdorf
Director Information TechnologyElectronic Sports League
Power savings of $648,000 per year for each group of 1000 servers converted!
Source: ESL Lab. Performance measured using esxtop & power meter with reference s/w stack running on Intel® Xeon® processors 7400. Power savings calculated based on 24x7x365 usage, ESL actual power rate & Yahoo $/€ exchange rate as of 2008-08-12. Power numbers and cost savings noted here are for server wall power only. No consideration of cooling power costs and other potential savings is included here. Actual performance and savings may vary.
Before After
Server 1P C2D 4P Xeon 7400
Cores 2 24
Game server processes
3 per CPU core
3 per VM; 4.5 per CPU core
VMs per box - 36
Game server processes per box
6 108
Users per box 72 1296
CPU Utilization 60-80% 75%
Power per physical server
350W 710W
Consolidation Ratio
18 1
Annual direct power cost
$731,000 per 1000 servers
$83,000 for 56 servers
Summary
Server Applications:
• Can We Virtualize Everything?
• New Technologies Allow Us To Virtualize More
• Question The Assumptions
• ROI Delivered
Do you have “non-virtualize-able” applications?
Evaluate the latest technologies.
Prove it!
Additional sources of information on this topic:
Intel Virtualization Developer Community http://softwarecommunity.intel.com/articles/eng/1424.htm
Intel Virtualization Technology http://www.intel.com/technology/virtualization/index.htm
Intel® Xeon® Processor 7300 Series http://www.intel.com/performance/server/xeon_mp/virtualization.htm?iid=SEARCH
More information on Intel® VT for Connectivity: www.intel.com/go/vtc
More information on Intel Networking solutions: www.intel.com/network
Intel® Virtualization Technology for Connectivity Tech Brief http://softwarecommunity.intel.com/isn/downloads/virtualization/pdfs/20137_LAD_VTc_Tech_Brief_r04.pdf
Virtual Machine Device Queues White Paper http://www.intel.com/technology/platform-technology/virtualization/VMDq_whitepaper.pdf
VMware ESX Server 3 Configuration Guide – Update 2 for ESX 3.5 http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_3_server_config.pdf Search for NetQueue/VMDq configuration info
Intelligent Queueing Technologies for Virtualization: Intel-VMware white paper http://download.intel.com/network/connectivity/products/whitepapers/Intel-VMware_VMDq_wp_May08.pdf
Intel 4-Processor Server System S7000FC4UR http://www.intel.com/products/server/systems/4-processor-s7000fc4ur/4-processor-s7000fc4ur-overview.htm
We want to hear from you…..
Join the conversation on Intel’s Open Port: The Server Room
Bring your questions to the “Ask an Expert” discussion forum
Share Best Practices with IT influencers
Engage with Intel on your DataCenter challenges
Check out -- communities.intel.com/community/server
Contact Info :Wm. Hank Lea:
william.h.lea@intel.comJosh Hilliker:
josh.hilliker@intel.com
Legal DisclaimerINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL® PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. INTEL PRODUCTS ARE NOT INTENDED FOR USE IN MEDICAL, LIFE SAVING, OR LIFE SUSTAINING APPLICATIONS.
Intel may make changes to specifications and product descriptions at any time, without notice.
All products, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice.
Intel, processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Dunnington and other code names featured are used internally within Intel to identify products that are in development and not yet publicly announced for release. Customers, licensees and other third parties are not authorized by Intel to use code names in advertising, promotion or marketing of any product or services and any such use of Intel's internal code names is at the sole risk of the user
Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance.
Intel, Intel Inside, Intel Xeon and the Intel logo are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2008 Intel Corporation.
Risk Factors
This presentation contains forward-looking statements that involve a number of risks and uncertainties. These statements do not reflect the potential impact of any mergers, acquisitions, divestitures, investments or other similar transactions that may be completed in the future. The information presented is accurate only as of today’s date and will not be updated. In addition to any factors discussed in the presentation, the important factors that could cause actual results to differ materially include the following: Demand could be different from Intel's expectations due to factors including changes in business and economic conditions, including conditions in the credit market that could affect consumer confidence; customer acceptance of Intel’s and competitors’ products; changes in customer order patterns, including order cancellations; and changes in the level of inventory at customers. Intel’s results could be affected by the timing of closing of acquisitions and divestitures. Intel operates in intensely competitive industries that are characterized by a high percentage of costs that are fixed or difficult to reduce in the short term and product demand that is highly variable and difficult to forecast. Revenue and the gross margin percentage are affected by the timing of new Intel product introductions and the demand for and market acceptance of Intel's products; actions taken by Intel's competitors, including product offerings and introductions, marketing programs and pricing pressures and Intel’s response to such actions; Intel’s ability to respond quickly to technological developments and to incorporate new features into its products; and the availability of sufficient supply of components from suppliers to meet demand. The gross margin percentage could vary significantly from expectations based on changes in revenue levels; product mix and pricing; capacity utilization; variations in inventory valuation, including variations related to the timing of qualifying products for sale; excess or obsolete inventory; manufacturing yields; changes in unit costs; impairments of long-lived assets, including manufacturing, assembly/test and intangible assets; and the timing and execution of the manufacturing ramp and associated costs, including start-up costs. Expenses, particularly certain marketing and compensation expenses, vary depending on the level of demand for Intel's products, the level of revenue and profits, and impairments of long-lived assets. Intel is in the midst of a structure and efficiency program that is resulting in several actions that could have an impact on expected expense levels and gross margin. Intel's results could be impacted by adverse economic, social, political and physical/infrastructure conditions in the countries in which Intel, its customers or its suppliers operate, including military conflict and other security risks, natural disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Intel's results could be affected by adverse effects associated with product defects and errata (deviations from published specifications), and by litigation or regulatory matters involving intellectual property, stockholder, consumer, antitrust and other issues, such as the litigation and regulatory matters described in Intel's SEC reports. A detailed discussion of these and other factors that could affect Intel’s results is included in Intel’s SEC filings, including the report on Form 10-Q for the quarter ended June 28, 2008.