tech•ed backstage 2010

David Connors – CodifyJorke Odolphi – MicrosoftBen Parker – Parker Tech

Sponsors

Agenda

Hello and Welcome!Wi-FiIPv4 / IPv6DemoNet & the IPv4 LegacyVendors and Vendon’ts and The Jiggle Test

Who’s who in the zoo?

Jack Morton WorldwidePlus a zillion other vendors for A/V, signage, food, etc.

JomablueTechnology Project Management & Logistics

CodifyArchitecture, engineering and hard nerd questions

Jorke / Parker Tech CollaborationDemoNet

Recent History of WiFi @ tech•edYear Venue Experience Solution

2004 Canberra Telstra Hotspot

2005 Gold Coast Convention and Exhibition Centre

Telstra Hotspot

2006 Sydney Convention Centre Excellent 8 Radios / Cisco 1232AG / Autonomous IOS

2007 Gold Coast Convention and Exhibition Centre

Excellent 16 Radios / Cisco 1232AG / Autonomous IOS

2008 Sydney Convention Centre EPIC FAIL due to venue sabotage

16 Radios / Cisco 1232AG / Autonomous IOS

2009 Gold Coast Convention and Exhibition Centre

Excellent 100 Radios / Cisco 1252AGN / Autonmous IOS

2010 Gold Coast Convention and Exhibition Centre

Excellent – World Class

100 Radios / Cisco 1252AGN / Lightweight IOS + Cisco WLC

Commissioning – We’ve heard of it!

WiFi Commissioning tips:• How to do it• Never let your standards drop• Take responsibility for the quality of your work

become a wifi expert in 15 minutes (apparently)

IEEE 802.3 EthernetCSMA/CDVarious physical layers and speedsHalf or Full DuplexUses copper wires and electricityInstallers use expensive Fluke testers and do their job properly

IEEE 802.11 WLANCSMA/CAVarious physical layers and speedsHalf DuplexUses luminiferous aether and juju magicInstallers talk gak at you, take all your money and don’t do their job properly

OMFG! Real World Tests!

Post Installation DOs• Run some file transfers!• With quiet RF and good gear you CAN get 100%

throughput ALL THE TIME – NO REALLY!!Post Installation DON’Ts• Check noise-floor and produce 200 pages of BS

from AirMagnet and run• Use iPerf for contrived/useless results

Demonstration

Wi-Spy DBx Analysing RF Utilisation

2006: Post Install Site Survey SCEC

2006: Post Install Real World TestsConfirmed:• ~2500 kilobytes per second from the Internet to

a WiFi client• Test done at each access point• Performed periodically throughout the event

2008: Same Process ... WTH?!Yep, same commissioning procedure and testing:• Same RF survey and post install commissioning• Same physical equipment• Same configuration scripts• What happened?

2008: Symptoms

With light-medium load:• Everything worked fine!• Great performance!With medium-heavy load (meal breaks):• Associations looked fine• Clients would get 169.254.x.x addresses• Clients with existing leases worked fine

DHCP DHCPDISCOVER

Client -> Broadcast

Locate all DHCP Servers on the Network

DHCPOFFER

DHCP Server(s) -> Client

Offer a lease

DHCPREQUEST

Client -> DHCP Server

DHCPACK

DHCP Server -> Client

One ... minor ... difference

• Technology Operation located in G01• Cable run too long for copper• Options

• A) Drill into car park, run fibre = ~$5K• B) Traverse the Procure gear in the venue

• Cheap = Bad

Example 1: Hall5DHCP request dispatched from Hall 5 South-WestNote:• The request is immediately repeated

on the port by the SCEC switch a few ten thousandths of a second later. There should be only one request every few seconds.

• The next slide shows what was received in G01.

Example 1: G01DHCP request received from Hall 5 South-West after SCEC core switch traversalNote:• The DHCP transaction IDs match

the previous slide so this is the same request.

• The data that leaves the SCEC edge does not match what was sent! There are many times more packets received in G01!

• In some cases there are 12 packets received in G01 when only one was sent by the client at the given time!

Example 2: Coalescing and delay of DHCP requests

Note DHCP requests are still duplicated in pairs

Initial request.

Second request made five seconds later

Third request made after eight seconds (due to DHCP back-off + slight randomisation as per the RFC)

Example 2: Coalescing and delay of DHCP requests

G01 receives multiple requests a few 10000ths of a second apart.

Turn off DHCP Snooping on SCEC Core

Correct behaviour: Non-duplicate DHCP Request / ACK pair

Correct behaviour: Non-duplicate DHCP Request / ACK pair

Correct behaviour: Non-duplicate DHCP Request / ACK pair

Correct behaviour: Non-duplicate DHCP Request / ACK pair

2009/2010 – GCCEC

Cisco WLC 4404 of DOOMExtensive RectificationNearly killed me

Described in detail at:http://www.msteched.com/2010/Australia/VOC208http://www.techedbackstage.net/tag/wireless/

OTW Customer Premise Equipment

Logical Overview

3550 12 G

3550 12 G

3550 12 G

3550 12 G

3550 12 G

2GBP

S LA

CP T

RUN

K

ADVA

OTW

Telstra Customer Premise Equipment

3750 48T

TID

IDF (there are 13 of these)

29701GBPS

1GBPS

Network Overview

IPv4

microsoftevent.localdomain demonet.localdomain show.localdomain

Over The Wire100mbps IP TransitIPv4 & IPv6 Transit

Telstra Internet Direct1gbps Layer 2 Service500mbps IPv4 Transit

500mbps PrivateLayer 2 Service

to Pipe NetworksDC 2, Creek Street

Public Address Space Network Segment

microsoftevent.localdomain Private Nework demonet.localdomain Private Nework show.localdomain Private Nework

50 x Cisco 1252AGNwifi Access Points

Cisco 5508Wireless LAN

Controller100BaseTX Wired Ports for:

* CommNet* Recharge Bars

DNS Resolver(Hyper-V Guest)

DNS Resolver(Hyper-V Guest)

DHCP Server(Hyper-V Guest)

DHCP Server(Hyper-V Guest)

\\services Print Spoolerand File Server(Hyper-V Guest)

DemoNetHyper-V Host

ServersAlso provides:

* DHCP* DNS

So these services areAD-integrated

rras1 rras2 rras3 rras5(Physical Server)

DNS Resolver(Hyper-V Guest)

DNS Resolver(Hyper-V Guest)

DHCP Server(Hyper-V Guest)

DHCP Server(Hyper-V Guest)

\\services Print Spoolerand File Server(Hyper-V Guest)

Delegates Exhibitors Untrusted Speakers Showcase SomewhatTrusted

Speakers Trusted

100BaseTX Wired port for each breakout room.

100BaseTX Wired port for each breakout room.

Cisco Router

IPv6

Public Address Space Network Segment

Cisco Router

2811

IPv4 & IPv6

REFER TO BEN PARKER’S CONTENTIPv4 & IPv6

IPv4 & IPv6

Why?

IPv4 & IPv6

hmmm 1. Turn multicast on to enable ipv6 RA

2. Ignore random ipv4 option that would lead you to believe this setting does

not affect v6

3. Enjoy the loss of fail-over if the WLC dies

because you are enabling ipv6’

POC

poc lab

100% Windows routing.statefull & stateless config

RA disabled even though enabled through gui…

netsh interface ipv6 set interface <int> advertise=enabled

netsh interface ipv6 set interface <int> advertisedefaultroute=enabled

ipnat.sys old

the event

/48 assigned from apnic2001:dfb::/48

/64 network segments for delegate/production/management

the eventCisco 2811’s replaced 2008r2 servers 2 hours before go live

Cisco Wireless LAN Controller blocks multicast by default

High end virtualised environment directly connected via ipv6 hosting over 120 virtual machines for demos.

A flat network with AD controllers live to the internet – your average MS sysadmin gets scared..

IPv4 regarded as legacy.

end result

traffic

IPv4 Traffic: 2TBHighly mixed

IPv6 Traffic: 35GBhttp: youtube/google/facebooksmb: misconfigured windows machines

CDN’s are still on v4….

World’s Smallest Violin

Vendors and Vendon’ts• Two Words• delete startup-configThe Jiggle Test120+ hour work weeks (here’s a hint: never work out your hourly rate)

DemoNET

24 x HP BL460c Blades (4 Enclosures, 2 Racks):• 8 x Highly Available Hyper-V Clusters (16 Servers)• 1 x Highly Available Hyper-V SP1 Beta Cluster (2

Servers)• 4 x Standalone Hyper-V• 2 x Windows 7Total 119 VMs:• 35 Showcase VMs• 72 VMs (36 Sessions)

The numbers

DemoNETThe layout

DemoNET

IPv6:• The world is accessible!• Public (Route) address spaceIPv4:• Legacy network• Public (Route) and Private (NAT) address space

The network (IPv6 is King)

DemoNETThe network – pt2

DemoNET

SYD/SIN/TPE > BNE• Over The Wire• Environment build (~30 days effort, 10 calendar

days)• Showcase and Speaker VM (imports)BNE > GCCEC• Showcase prep• Go live

Logistics

Question and Answer

Question and Answer TimeSession Title

Related Content

Related Content

http://www.techedbackstage.net/http://www.msteched.com/2010/Australia/VOC208

Thanks For Listening!

Be sure to submit your feedbackif you want to be in the draw to

win the Xbox 360 and other prizes!

Sponsors