Date post: | 17-Jan-2017 |
Category: |
Software |
Upload: | amazon-web-services |
View: | 338 times |
Download: | 2 times |
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Shahbaz Alam, AWS Professional Services
September 2016
Another Day, Another Billion
Packets
Deja Vu
We have the cloud
Amazon
EBS
Amazon
RDS
Amazon
ElastiCache
Amazon
Redshift
Amazon EC2 Elastic Load
Balancing
We have customers
Some customers have existing data centers
Customers want to make their datacenters
work with the cloud
???
Whiteboard engineering
Amazon
EBS
Amazon
RDS
Amazon
ElastiCache
Amazon
Redshift
Amazon
EC2
Elastic Load
Balancing
EC2 as it was
10.44.12.4 10.44.12.5
10.44.92.1710.44.12.27
10.108.6.4
Why that doesn’t work
192.168.0.0/16
Routing Table
• 192.168.0.0/16: stay here
• 10.44.12.4/32: AWS
• 10.44.92.17/32: AWS
• 10.108.6.4/32: AWS
10.44.0.0/16
10.44.12.4 10.44.12.5
10.44.92.1710.44.12.27
10.108.6.4
Design Requirements
• Customer selected IP addresses
• Route aggregation for external connectivity
• Conformance with existing network designs
172.31.0.0/18
192.168.0.0/16
Routing Table
• 192.168.0.0/16: stay here
• 172.31.0.0/18: AWS
172.31.1.0/24 172.31.2.0/24
172.31.1.7
172.31.1.8
172.31.1.9
172.31.2.12
172.31.2.51
Amazon Virtual Private Cloud (VPC)
This is just virtual networking!
Subnet ~= VLAN
VPC ~= VRF (virtual routing and forwarding)
But…
Scaling challenges
VLAN ID space is constrained
• 12 bits => 4096 total VLANs
VRF support is constrained
• Large routers => 1-2 thousand VRFs
Fixed ratio of VLANs:VRFs
Router and capacity dimensions
Big Router
Data Plane
Control
Plane
Big Router
Data Plane
Control
Plane
An example
Average router configuration line: 50 chars
Config per VPC: 10 lines
Subnets per VPC: 4
Config per subnet: 5 lines
Total VPCs: 2,000
Config size: 3 MB
But…
This doesn’t scale
• 12 bit VLAN ID = 4096 VLANs (not
enough)
• BIG routers support 4,000 VRFs
($200k+)
Large VLANs make Network Engineers cry
Tied to vendor bugfix cycles (6 months +)
BIG virtual routers are built by few
companies
Interoperability of advanced features is
marginal
$$$
Silos of capacity (illustrative)
A
C
B
FE
D
G
A AA
A
B
C
B B
B B
C
D
F FF
D
D
B
G G
/4 /4
/40 /40
0
0
0
0
1324 132
C
G G
3 27
D DD
9910
F F F F F
1815 40
BB B B B
BB B B B
BB B B B
B B
Functional requirements
• Scale to millions of environments the size of
Amazon.com
• Any server, anywhere in a region can host an instance
attached to any subnet in any VPC
Let’s review: L2 – Ethernet
10.0.0.2
10.0.0.3
L2 Src: MAC(10.0.0.2)
L2 Dst: ff:ff:ff:ff:ff:ff
ARP Who has
10.0.0.3?
The switch floods the
ARP request out all
ports
Ethernet Switch
L2 Src: MAC(10.0.0.3)
L2 Dst: MAC(10.0.0.2)
ARP 10.0.0.3 is at
MAC(10.0.0.3)
The switch snoops the
ARP response and
learns the port for
MAC(10.0.0.3).
L2 Src: MAC(10.0.0.2)
L2 Dst: MAC(10.0.0.3)
L3 Src: 10.0.0.2
L3 Dst: 10.0.0.3
ICMP/TCP/UDP/…
Let’s review: L3 – IP routing
10.0.0.2
10.0.1.3
L2 Src: MAC(10.0.0.2)
L2 Dst: ff:ff:ff:ff:ff:ff
ARP Who has
10.0.0.1?
Ethernet Switch
L2 Src: MAC(10.0.0.1)
L2 Dst: MAC(10.0.0.2)
ARP 10.0.0.1 is at
MAC(10.0.0.1)
L2 Src: MAC(10.0.0.2)
L2 Dst: MAC(10.0.0.1)
L3 Src: 10.0.0.2
L3 Dst: 10.0.1.3
ICMP/TCP/UDP/…
RouterEthernet Switch
L2 Src: MAC(10.0.1.1)
L2 Dst: MAC(10.0.1.3)
L3 Src: 10.0.0.2
L3 Dst: 10.0.1.3
ICMP/TCP/UDP/…
VPC Concepts
Server 192.168.0.3
Server 192.168.0.4
…
Server 192.168.1.3
Server 192.168.1.4
…
10.0.0.2
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
Server:
Physical hypervisor
in an Amazon data
center
Instance:
Amazon EC2
instance owned by a
customer
VPC:
Amazon Virtual
Private Cloud
owned by a
customer
VPC ID:
Identifier for a VPC
such as vpc-
1a2b3c4d
Mapping Service:
Distributed lookup
service. Maps VPC
+ Instance IP to
server
L2 - VPC
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
L2 Src: MAC(10.0.0.2)
L2 Dst: ff:ff:ff:ff:ff:ff
ARP Who has
10.0.0.3?
L2 Src: MAC(10.0.0.3)
L2 Dst: MAC(10.0.0.2)
ARP 10.0.0.3 is at
MAC(10.0.0.3)
Src: 192.168.0.3
Dst: Mapping Service
Query:
Blue 10.0.0.3
Src: Mapping Service
Dst: 192.168.0.3
Reply:
Host: 192.168.1.4
MAC: MAC(10.0.0.3)
10.0.0.2
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
10.0.0.2
…
L2 Src: MAC(10.0.0.2)
L2 Dst: MAC(10.0.0.3)
L3 Src: 10.0.0.2
L3 Dst: 10.0.0.3
ICMP/TCP/UDP/…
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.1.4
Src: 192.168.1.4
Dst: Mapping Service
Validate:
Blue 10.0.0.2 is at
192.168.0.3
Src: Mapping Service
Dst: 192.168.1.4
Mapping valid:
Blue 10.0.0.2 is at
192.168.0.3
L2 - VPC
…
VPC isolation
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
10.0.0.2
Src: 192.168.0.4
Dst: Mapping Service
Query:
Grey 10.0.0.3
L2 Src: MAC(10.0.0.4)
L2 Dst: ff:ff:ff:ff:ff:ff
ARP Who has
10.0.0.3?
VPC isolation
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
10.0.0.2
Src: 192.168.0.4
Dst: Mapping Service
Query:
Blue 10.0.0.3
L2 Src: MAC(10.0.0.4)
L2 Dst: ff:ff:ff:ff:ff:ff
ARP Who has
10.0.0.3?
192.168.0.4 is not
hosting any instances
in VPC Blue.
Mapping Denied
Alarm Raised
VPC isolation
Server 192.168.0.3
Server 192.168.0.4
…
Server 192.168.1.3
Server 192.168.1.4
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
10.0.0.2
…
L2 Src: MAC(10.0.0.4)
L2 Dst: MAC(10.0.0.3)
L3 Src: 10.0.0.4
L3 Dst: 10.0.0.3
ICMP/TCP/UDP/…
VPC: Blue
Src: 192.168.0.4
Dst: 192.168.1.4
Src: 192.168.1.4
Dst: Mapping Service
Validate:
Blue 10.0.0.4 is at
192.168.0.4
Src: Mapping Service
Dst: 192.168.1.4
Mapping invalid!
192.168.1.4 does not
deliver the packet to
the instance.
Alarm Raised.
L3 - VPC
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
10.0.1.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
L2 Src: MAC(10.0.0.2)
L2 Dst: ff:ff:ff:ff:ff:ff
ARP Who has
10.0.0.1?
L2 Src: MAC(10.0.0.1)
L2 Dst: MAC(10.0.0.2)
ARP 10.0.0.1 is at
MAC(10.0.0.1)
Src: 192.168.0.3
Dst: Mapping Service
Query:
Blue 10.0.0.1
Src: Mapping Service
Dst: 192.168.0.3
Reply:
Host: Gateway
MAC: MAC(10.0.0.1)
10.0.0.2
L3 - VPC
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
10.0.1.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
Src: 192.168.0.3
Dst: Mapping Service
Query:
Blue 10.0.1.3
Src: Mapping Service
Dst: 192.168.0.3
Reply:
Host: 192.168.1.4
MAC: MAC(10.0.1.3)
10.0.0.2
L2 Src: MAC(10.0.0.2)
L2 Dst: MAC(10.0.0.1)
L3 Src: 10.0.0.2
L3 Dst: 10.0.1.3
ICMP/TCP/UDP/…
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.1.4
Src: 192.168.1.4
Dst: Mapping Service
Validate:
Blue 10.0.0.2 is at
192.168.0.3
Src: Mapping Service
Dst: 192.168.1.4
Mapping valid:
Blue 10.0.0.2 is at
192.168.0.3
L2 Src: MAC(10.0.1.1)
L2 Dst: MAC(10.0.1.3)
L3 Src: 10.0.0.2
L3 Dst: 10.0.1.3
ICMP/TCP/UDP/…
Caching
Server 192.168.0.3
Server 192.168.0.4
Server 192.168.1.3
Server 192.168.1.4
…
10.0.0.2
10.0.0.3
10.0.0.4
10.0.0.4
10.0.0.2
10.0.0.5
10.0.0.3
Mapping Service
L2 Src: MAC(10.0.1.1)
L2 Dst: MAC(10.0.1.3)
L3 Src: 10.0.0.2
L3 Dst: 10.0.1.3
ICMP/TCP/UDP/…
10.0.0.0/18
172.16.0.0/16
10.0.0.0/24 10.0.1.0/24
10.0.0.7
10.0.0.8
10.0.0.9
10.0.1.12
10.0.1.51
VPC: Blue
Src: 192.168.0.3
Dst: ???
L3 Src: 10.0.0.7
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
Getting home (or anywhere, really)
Edges
Server 192.168.0.3
Server 192.168.0.4
Edge 192.168.4.3
Edge 192.168.4.4
10.0.1.3
10.0.0.4
10.0.0.2
Mapping Service
10.0.0.2
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
Host 10.0.0.4 192.168.0.4
Host 10.0.1.3 192.168.0.4
…
172.16.0.0/16 Edge 192.168.4.3
…
Edges (three different ones) – VPN
Edge 192.168.4.3VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
IPSEC Stuff
Src: 54.68.100.245
Dst: 205.251.242.54
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
Edges (three different ones) – AWS Direct Connect
Edge 192.168.4.3VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
802.1Q VLAN Tag
Src: 54.68.100.245
Dst: 205.251.242.54
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
Edges (three different ones) – Internet
Edge 192.168.4.3VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 176.32.96.190
ICMP/TCP/UDP/…
L3 Src: 10.0.0.2
L3 Dst: 176.32.96.190
ICMP/TCP/UDP/…
54.148.157.46
Edges (three different ones)
VPNEdge 192.168.4.3
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
IPSEC Stuff
Src: 54.68.100.245
Dst: 205.251.242.54
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
Direct ConnectEdge 192.168.4.3
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
802.1Q VLAN Tag
Src: 54.68.100.245
Dst: 205.251.242.54
L3 Src: 10.0.0.2
L3 Dst: 172.16.14.17
ICMP/TCP/UDP/…
InternetEdge 192.168.4.3
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.3
L3 Src: 10.0.0.2
L3 Dst: 176.32.96.190
ICMP/TCP/UDP/…
L3 Src: 54.148.157.46
L3 Dst: 176.32.96.190
ICMP/TCP/UDP/…
Image credit: Wikipedia
https://en.wikipedia.org/wiki/1918_Eighth_Avenue
A brief diversion – Fun Fact
Back to our regularly scheduled program…
Amazon S3
172.31.0.0/18
172.31.1.0/24 172.31.2.0/24
172.31.1.7 172.31.2.12
Amazon S3 endpoints
172.31.0.0/18
172.31.1.0/24 172.31.2.0/24
172.31.1.7 172.31.2.12
Server 192.168.0.3
Server 192.168.0.4
Edge 192.168.4.3
Edge 192.168.4.4
10.0.1.3
10.0.0.4
10.0.0.2
10.0.0.2
L3 Src: 10.0.0.2
L3 Dst: 54.231.33.89
TCP/HTTP/…
VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.4
L3 Src: 10.0.0.2
L3 Dst: 54.231.33.89
TCP/HTTP/…
EdgesMapping Service
Host 10.0.0.4 192.168.0.4
Host 10.0.1.3 192.168.0.4
…
172.16.0.0/16 Edge 192.168.4.3
S3.us-east-1 Edge 192.168.4.4
…
A new edge – S3 endpoint
Edge 192.168.4.4VPC: Blue
Src: 192.168.0.3
Dst: 192.168.4.4
L3 Src: 10.0.0.2
L3 Dst: 54.231.33.89
TCP/HTTP/…
VPC Endpoint 1a2b3c4d
Src: 54.68.100.245
Dst: 54.231.33.89
L3 Src: 10.0.0.2
L3 Dst: 54.231.33.89
TCP/HTTP/…
Endpoints and policy
172.31.0.0/18
172.31.1.0/24 172.31.2.0/24
172.31.1.7 172.31.2.12
{
"Statement": [
{
"Sid": "Access-to-specific-bucket-only",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::my_secure_bucket",
"arn:aws:s3:::my_secure_bucket/*"]
}
]
}
{
"Statement": [
{
"Sid": "Access-to-specific-VPC-only",
"Principal": "*",
"Action": "s3:*",
"Effect": "Deny",
"Resource": ["arn:aws:s3:::my_secure_bucket",
"arn:aws:s3:::my_secure_bucket/*"],
"Condition": {
"StringNotEquals": {
"aws:sourceVpc": "vpc-111bbb22"
}
}
}
]
}
Nov 10, 2010
172.31.0.0/18
172.31.1.0/24 172.31.2.0/24
172.31.1.7
172.31.1.8
172.31.2.12
172.31.2.51
VPC as a platform
Simple Complex
Limited Flexible
EC2 VPC
VPC pricing
Cost per VPC: $0.00
Cost per subnet: $0.00
Upcharge per instance: $0.00
172.31.0.0/18
172.31.1.0/24 172.31.2.0/24
172.31.1.7
172.31.1.8
172.31.1.9
172.31.2.12
172.31.2.51
Default VPC
Simple Complex
Limited Flexible
EC2 - VPC
VPC CIDR 10.1.0.0/16
Availability Zone A Availability Zone B
Public Subnet Public Subnet
Private Subnet Private Subnet
Instance A10.1.1.11 /24
Instance B10.1.2.22 /24
Instance C10.1.3.33 /24
Instance D10.1.4.44 /24
10.1.1.0/16
10.1.2.0/16
10.1.3.0/16
And Today…
Public Subnet
Remember to complete
your evaluations!