16-Feb-05 DREN IPv6 Update 1

DREN IPv6 Implementation Update

Joint Techs WorkshopFeb 2005

Salt Lake City, UT

Ron BroersmaDREN Chief Engineer

High Performance Computing Modernization Programron@spawar.navy.mil

16-Feb-05 DREN IPv6 Update 2

Context• Historical

– 2001 – DREN IPv6 testbed• Wide area• Dedicated hardware – 10 “core” nodes.• Native IPv6 over partial ATM mesh

– 2003 – DoD and IPv6• DoD CIO issues memorandum to transition by 2008• DREN chosen as the DoD “pilot implementation”

– 2003/2004 – DoD “pilot” on DREN production network• dual stack, native, running on production DREN network

– 2004/2005 – additional efforts• site deployment, multicast, DHCP/DNS, mobility

• Within DoD…– Each of the services (Army, Navy, Air Force) developing their own

transition plans for the “operational networks”.• Most will not begin implementation for a year or more• Most will not be complete until after 2008

– DREN is DoD’s “research network”, and is transitioning now.• Chartered to support the DoD HPC community, and other R&D

organizations.

16-Feb-05 DREN IPv6 Update 3

DREN Today

• 10 “core nodes” on OC-192 backbone (CONUS), with OC-12 extensions to Hawaii and Alaska.

• About 100 sites (“Service Delivery Points”), connected at DS-3 to OC-48 rates.

• IPv4 unicast and multicast, IPv6 unicast, and ATM services now.

• Dual IPv6 networks (“testbed”, and “production”)

• “jumbo-clean” (i.e. 9K MTU everywhere)• Multiple security levels.

– Both unclassified and classified networks

16-Feb-05 DREN IPv6 Update 4

DREN “production” network

16-Feb-05 DREN IPv6 Update 5

DRENv6 “testbed”Logical Topology

Dayton

San Diego

Albuquerque

Wash D.C.

Stennis

Vicksburg

Aberdeen

ATM PVC (OC-3)

tunnel

HICv6

(Hawaii)

GlobalCrossing

HurricaneElectric

LAVAnet

SPRINT

vBNS+

6TAP

SSC CharlestonSSAPAC

SSC San Diego

WCISD

AOL

NRL

ARLWPAFB

ERDC

NAVO

C&W

Cisco

NTTComVerio

AFRLKirtland AFB

Abilene

SD-NAPSDSC

Core Router

“site”

IXP

ISP orBGP Neighbor

FIX-West Abilene

HP

AIX-v6

TIC

JITC

Tunnel broker

16-Feb-05 DREN IPv6 Update 6

DREN IPv6 philosophy

• Push the “I believe” button, and turn on IPv6 everywhere to see what works (and what doesn’t)

• Do it in a production environment– can get away with this in an R&D

environment, but not on operational networks.

• Go native. (no tunnels)• Even if the world doesn’t convert for

years, R&D environments need it now.• Figure out how to deploy IPv6 to the

rest of DoD in the future.

16-Feb-05 DREN IPv6 Update 7

2003/2004 DREN IPv6 Initiative

• DoD IPv6 Pilot network• Goals for 2004

1. IPv6 enabled DREN infrastructure (all Service Delivery Points, the Wide Area Network, the NOC). Done

2. Facilitate IPv6 deployment into infrastructure at HPC user sites and DREN user sites. Done

3. IPv6 enabled HPCMPO, HPCMP funded assets and services, HPCMP user community support applications, selected user application candidates. Partial completion

4. Performance and Security as good as existing IPv4 service. Done

5. Provide product feedback, lessons learned, published via web. Done

16-Feb-05 DREN IPv6 Update 8

Some things we learned

• Many security components are missing.• 1 + 1 > 2

– managing 2 IP networks (IPv4, IPv6) can be more than double the complexity due to new interactions. Making topologies congruent can minimize this effect.

• Site deployment – little priority for IPv6• Lack of applications support

16-Feb-05 DREN IPv6 Update 9

Lack of Security Features (Examples)

• Router Access Control Lists (ACLs)– Juniper doesn’t support “tcp established”

• Vulnerability Assessment (Scanners)– ISS doesn’t support IPv6 and has no published plans to do so.– NESSUS doesn’t support IPv6 (yet)

• Intrusion Detection Systems– If we want IPv6 support, we have to add it ourselves.– Juniper port mirroring doesn’t support IPv6

• IPSEC– Missing in most IPv6 implementations– Juniper ASPIC doesn’t support IPv6 (until much later)

• Firewalls– Until recently, no production quality IPv6 support– Netscreen (Juniper):

• no OSPFv3, only RIP• IPv6 support only available in certain products

It is crucial that IPv6 products have equivalent functionality to the IPv4 world

16-Feb-05 DREN IPv6 Update 10

DoD Security Model

• “Defense in Depth”– Protections at

multiple levels

• Problem: How to securely deploy IPv6 in DoD without these components.

InternetInternet

WANWAN

LANLAN

S

IDSACL

Firewall

IDS

ACL

Scanners

16-Feb-05 DREN IPv6 Update 11

Overcoming the security issue (workaround)

• Use DRENv6 testbed for transit to Internet– use to peer with rest of IPv6 enable Internet and other testbeds– continue to operate as an “untrusted” IPv6 network

• Enable IPv6 on new DREN2 (MCI) production network.– Dual stack everywhere.

• Establish trusted gateways between v6 enabled DREN2 and the DRENv6 testbed– Upgrade HPC Network Intrusion Detection Systems (NIDS) to be

v6-compliant, monitored by the HPC Computer Emergency Response Team (CERT), and install at the trusted gateways.

– Install v6 version of standard DREN v4 Access Control Lists (ACLs) to protect pilot network to same level as IPv4 production network.

• DREN customers receive “safe” native IPv6 service via existing service delivery point (SDP), in parallel with IPv4 service.

16-Feb-05 DREN IPv6 Update 12

DREN IPv6 transition architecture – FY04

DRENv6 (Testbed)DRENv6 (Testbed)

DREN2 (Production / Pilot)DREN2 (Production / Pilot)sdp.arlapgsdp.sandiego

sdp.erdc

SSCSDERDC

ARL-APG

NIDSv6NIDSv6 NIDSv6

v6 ACLv6 ACL

v6 ACL

To 6bone, Abilene, and other IPv6 enabled ISPs IPv6 demonstrations (Moonv6)

Dual stack IPv4 and IPv6 wide area infrastructure

sdpsdp

sdp

Type “A” (IP) production service to DREN sitesIPv4 and IPv6 provided over the same interface

Testbed atDREN site

Testbed atDREN site

Native IPv6 backbone

links run native IPv6 where possible, otherwise tunnelled in IPv4

Goal: As secure asthe IPv4 backbone

16-Feb-05 DREN IPv6 Update 13

Site Security Solution(Example – SPAWAR)

• SPAWAR Intrusion Detection System (IDS) modified to support IPv6

• Netscreen Firewall operating “beta” release with IPv6 support in parallel with production firewall.

DREN2 (Pilot)

DREN2 (Pilot)

SPAWARBorder router(Juniper M20)

Netscreen 2000Firewall

to LAN

IPv4 unicast andmulticast services+ IPv6 unicast

Netscreen 208Firewall

switch

IPv4 IPv6

IDS

ProductionFirewall

WAN

IPv6 FirewallNote: Netscreen (Juniper) now hasmainstream IPv6 support for some models.

16-Feb-05 DREN IPv6 Update 14

Plans for 2004/2005• Continued IPv6 deployment into site

infrastructure, and site upgrades.– includes training, and site visits

• Upgrade HPC applications to IPv6• Additional external peering• IPv6 multicast (both networks)• DHCPv6/DNS experiments

– what is best design model for DoD sites?

• Mobility experiments• Overcoming security challenges• BGP confederations• IPv6 on S/DREN

16-Feb-05 DREN IPv6 Update 15

New challenges impacting IPv6 implementation efforts

• Encrypt DREN backbone– Full IPSEC mesh between all DREN sites– Using Juniper Adaptive Services (AS) PIC.– Surprise: Doesn’t support IPv6.

• still 6 months away (JunOS 7.4?)

• BGP confederations – improved unicast and multicast routing.– CONUS, Hawaii, Testbed

• OC-48 sites.– IPSEC Encryption is the hard part. Trying to

do it with Netscreen 5400s using 10GbE interfaces. But they weren’t jumbo-clean.

16-Feb-05 DREN IPv6 Update 16

IPv6 multicast• Initiative:

– turn up IPv6 multicast on both nets (testbed, production)• PIM, MLDv2, MBGP, SSM, Embedded RP• apps: diag tools like beacon, mping, mtrace• then try other apps (vic, rat, …)

• Status (work in progress)– Testbed: Done

• routers all upgraded – IOS 12.3(11)T• Static RP

– Production: Some initial configuration completed– Setting up beacon infrastructure within DREN

• Some Issues– no MSDP, so use SSM or Embedded-RP between domains– Embedded RP is fairly new (i.e. need JunOS 7.0 or later)– many tools don’t operate over SSM (example: beacon)

• hard to do cross-domain testing– no MLDv2 in WinXP, broken in old Linux, Solaris.

16-Feb-05 DREN IPv6 Update 17

IPv6 DHCP/DNS• Problem:

– for sites that manually register everything in DNS today, this isn’t going to work well in IPv6.

– How to leverage auto-configuration capabilities, yet stay within local policies.

• Initiative:– what model and tools to recommend to DoD sites?

• test various implementations, and see what works

• Status (work in progress):– playing with open-source (sourceforge) DHCPv6

implementation• Some Issues:

– no DNS update in sourceforge DHCPv6– ISC DHPC (what most sites use) doesn’t do IPv6– WinXP doesn’t do DHCPv6

16-Feb-05 DREN IPv6 Update 18

Site infrastructure work

• IPv6 firewall, IDS, ACLs• LAN infrastructure (San Diego example)

– Backbone upgrade (Foundry core/dist’n/edge)– BigIron MG8

• 10GbE backbone– (low power)

• line rate IPv4 and IPv6 requirement– recent test – 6 x 10G IPv6 – ran at line rate

– Issues:• Foundry: NUD seems broken – loses initial packets of

new connections.• Foundry: IPv6 PIM-SM not supported (yet)• No production 10Gb firewall capable of IPv6 and jumbo.

– have beta netscreen hardware