Date post: | 04-Jan-2016 |
Category: |
Documents |
Upload: | shanon-crawford |
View: | 215 times |
Download: | 0 times |
Network Architecture
Gary Buhrmaster
ST&E Readiness ReviewMay 14th, 2007
Work supported by U. S. Department of Energy contract DE-AC03-76SF00515
Network Philosophy
Support getting the science done (safely) The science is the thing
Simplicity (where possible) Limit vendors, technologies used Leverage existing SCCS staff expertise
Redundancy (where appropriate) SCCS is not staffed for 24/7 coverage “Throwing smart (dedicated) people at issues”
works as long as you do not throw them too often
Overview
SLAC administers globally routed network space of 134.79.0.0/16 “SLAC” address space Visitor and RAS subnets IPv6 (test) subnet
A number of internal private subnets for control systems, isolated systems, batch farms Accelerator, SSRL, IR2, SCCS
Overview
Hardware Vendors: Cisco, Nokia ~300 Layer 2 (capable) devices ~50 Layer 3 (capable) devices ~20 Enforcement (firewall/filter) devices Many devices are categorized as more than one
swouters/frankenrouters (not all swouters are used as L2/3) what is an infiniband “switch” (it has routing in it…)
Misc. appliances (WLSE (HP), EndRun) ~15 support systems (logging, monitoring, etc.)
Sun/Dell – systems managed by the systems group
Overview
Physical instantiation ~70 buildings
Some buildings have numerous switches (some none) klystron gallery, computer center, SSRL
~200 VLANS Switched network design Some buildings have multiple subnets/vlans Some vlans are in multiple buildings, some in only one
Some in only one switch router to router connections, span monitoring…
Some internally used by devices
Staffing
Network Engineering Manage/Configure/Monitor network devices Five FTEs
Network Research Primarily research activities
But operationally focused (not just blue sky), which is leveraged to support SLAC and HEP/BES activities (especially WAN performance issues)
Staffing (outside of Network group) Network Operations
Reports to SCCS Operations Physical installation/support Five FTEs
Netops also coordinate with CEF staff and contractors for some installations (cable pullers, bulk fiber installation and termination, etc.)
Staffing (outside of Network group) Security group
Responsible for overall security policies and approvals
Apply approved policies to the Cisco enforcement devices
Windows group Apply approved policies to the Checkpoint
enforcement devices Systems group
Maintain the Unix network support systems
SLAC Speak
IFZ – Internet Free Zone At least some part of every network is blocked
from offsite network access Printers, Batch nodes, Network devices, “problematic”
devices (i.e. SBCs/IOCs)
SFZ – SLAC Free Zone Some special networks (controls) are accessible
only from their local networks IR2, MCC
SLAC Speak
RouterBlock Layer 3 forward and uRPF blocking (advertise
the /32 addresses into routing table to null route device at the router(s))
EPN – “Extremely Private Network” Elevated level protections (the “PII” place)
EPN(1) (original design), EPN2 (revised design)
CANDO – Computer And Network Database in Oracle (?)
Database of record for IP addresses/systems
……………………………………………………
Big (dense) Picture
Border
MCCIR2 SSRL BSDEPN
VPN “Special”NetMgmt
Farm Netrsch
Infra Campus
Core
“Internet”IPv6
visitor
But still simplified
Drill down (Layer 3 view)
Network segmentation Enclaves
SLAC, accelerator…. Functional/Physical
research yard, visitor network, decnet Performance/Availability
batch farm, network research
IPv6 Network
Dipping a toe in the (IPv6) water its cold and lonely there
External to SLAC network One web server
was originally proposed to be named VVVVVV
ESnetBAMANrtr-ipv6
IPv6 Network
WWW
Visitor (& RAS) Network
External to SLAC network (no trust) Wireless access is only on visitor network Client only support (block servers)
ESnet
BAMAN
Visitor Network
Border Network
Border enforcement device is a filtering router ACLs block ports <1024 (except to allowed hosts), and
various special ports (X, netbus, backoriface, …)
ESnet
Stanford
CENICInternet2
BAMANBorderrouter
Infrastructure Services
Centrally administered servers Windows/Unix infrastructure services
Unix & Windows infrastructure – DNS, Kerberos, AFS, AD, file servers, web services, email, ….
IFZ and where possible Most exceptions to port < 1024 filters are to these
servers (web, email, kerberos)
SLAC Network“Nethub/IFZ/IFZ-Lite”
B050(2nd floor)
Campus
Most staff/engineers/scientists are connected to one of the “PUB” networks Legacy workgroup allocations (based on “yellow
cable”) have changed to physical location allocations (trying to avoid flat earth operations)
Campus
Campus Distribution
Access (many buildings)
Farm
Batch resources for scientific discovery Most resources are IFZ
Exceptions for external data transfer systems, and scientific login systems
Many resources are (policy (i.e. netgroup)) limited to be used only from other batch systems
Different Availability/Performance needs
SLAC Network
“Farm” networks
batch systems
Campus
BaBar / IR2
IR2 has four subnets one public general purpose subnet, one IFZ
subnet (local compute farm), one SFZ subnet (dedicated SBCs and detector subsystems) with EPICs gateway, and isolated device control
Intention is that these networks/systems can operate independently from SCCS
mcc
Farm
Accelerator (MCC)
Accelerator network has four subnets One public general purpose subnet (slclavc), two “slac free”
subnets (leb, slcc) for control systems, and one isolated subnet (pep)
Use of multi-homed controls systems (VMS) for access to isolated networks devices
Intention is that these networks/systems can operate independently from SCCS
IR2
SLAC Network
Network Management
Network monitoring and configuration management (BAM - Backup and Monitoring) SNMP (via acls on network devices) only respond
to requests from the management network hosts ACLs protect appliances/APs (bastion hosts) Systems are limited access
SLAC Network Network Management and monitoring networks
Network Research
Network Research activities Isolated to allow local experimentation
ex: tsunami multicast
Systems are maintained the same as other systems on site
Systems are limited login, sponsored users
SLAC Network Research network
SSRL
SSRL manages their own network equipment and configurations, including their own firewall implementations to protect their control and experimental systems A later presentation will discuss SSRL
BSD (EPN(1))
EPN(1) Air Gap possibility Extensive filtering Users access PeopleSoft via Citrix More details in later presentation
rtr-bsdnet
bsd-epnbsd
SLAC net
bsd-dmz
EPN2
Revised approach based on new PS arch Multiple DMZ nets (web servers), Backend nets
(app servers, DBs) In realty, collapsed firewalls
Details in later presentation
SLAC Network
DMZsBackend
VPN
VPN (GRE/IPSEC) only to official servers Windows PPTP/L2TP VPN server Discouraged (use Citrix where possible) Firewall/filters
Block RPC, NFS, CIFS except to approved servers, & NetBus, BackOriface, etc.
SLAC Network VPN Servers
“Special” subnet(letts)
A few networks specially protected due to inability to maintain the systems, or certified configurations Ex: GLAST Clean Room, PCD, HVAC Group responsible for equipment purchase, SCCS
maintains the devices/configurations
SLAC Network
SLAC Network
SLAC Network
Procedures/Policies
Device connection policy Devices need to be in CANDO
Network equipment Users are not to install switches/routers/hubs
Wireless No wireless on the SLAC networks Devices installed/coordinated by SCCS
Network protections
Dedicated subnet for network management Network devices are IFZ SNMP restricted to network management subnet
SSH on all but a few legacy devices Finally got funding to upgrade the last few
Disable ports not allocated on switches No devices on native .1q vlan WLSE used for rogue access point detection
Network protections
Restricted physical access to “core” devices (Building 050 OmniLock door access)
Routing/switching best practices no ip unreachable, BGP passwords, schedule
allocate, no source route, …. Strong working relationship with upstreams
Network Intrusion Detection
Primarily log and netflow based Central logging and analysis
“Significant” events cause paging Netflow detects many scanners (and P2P)
Collected for both internal and external traffic “scanning” detection catches (SMTP) bots in “real time”
And the occasional “special” user Extremely useful for incident analysis
Discussion?
Obligatory final slide to avoid “End of slide show” artifact