Date post: | 07-Feb-2017 |
Category: |
Software |
Upload: | stephen-gordon |
View: | 32 times |
Download: | 0 times |
DELIVERING OPENSHIFT ON OPENSTACK FOR PERFORMANCE AND SCALESTEVE GORDON (@xsgordon)
GOAL
● 1000 OpenShift Container Platform nodes; hosted on● 300 physical Red Hat OpenStack Platform nodes
2
3
APPLICATION CONTAINERS
1. Code: mysqld
2. Configuration: /etc/my.cnf
3. Data: /var/lib/mysql
4. Other stuff :-)
CODE, CONFIGURATION, DATA
4
EXPOSITION OF RESOURCES
5
CONSUMPTION OF RESOURCES
OPENSHIFT ON OPENSTACK
6
Architectural tenets:
● Technical independence● Contextual awareness● Avoiding redundancy● Simplified management
Reference architecture:● red.ht/2ibNmvX
PERFORMANCE AND SCALE TESTING
7
WHERE TO TEST?
8
The CNCF cluster is made up of 1000 nodes deployed at Switch, Las Vegas by Intel for the use of the CNCF community:
Compute Node Spec● 2x Intel E5-2680v3 12-core● 256GB RAM● 2x Intel S3610 400GB SSD● 1x Intel P3700 800GB NVMe PCIe
SSD● 1x QP Intel X710
Storage Node Spec● 2x Intel E5-2680v3 12-core● 128GB RAM● 2x Intel S3610 400GB SSD● 10x Intel 2TB NLSAS HDD● 1x QP Intel X710"
Got a nefarious plan for taking over the world using CNCF related open source projects? Head to https://github.com/cncf/cluster.
WHAT TO TEST?
Goal: 1000 OpenShift Container Platform nodes, on 300 Red Hat OpenStack Platform nodes● Push deployment to its limit, identify:
○ Bottlenecks,○ Config changes, and○ Best practices
● Document, file, and fix issues as appropriate.
9
ARCHITECTURE
10
● OpenStack UnderCloud
ARCHITECTURE
11
● OpenStack UnderCloud● OpenStack Overcloud control
plane
ARCHITECTURE
12
● OpenStack UnderCloud● OpenStack Overcloud control
plane● OpenStack Overcloud compute
ARCHITECTURE
13
● OpenStack UnderCloud● OpenStack Overcloud control
plane● OpenStack Overcloud compute● OpenShift/Kubernetes master node
VMs● OpenShift routing, registry, and
metrics● OpenShift/Kubernetes minions● Load Generators
ARCHITECTURE
14
● OpenStack UnderCloud● OpenStack Overcloud control
plane● OpenStack Overcloud compute● OpenShift/Kubernetes master node
VMs● OpenShift routing, registry, and
metrics● OpenShift/Kubernetes minions
ARCHITECTURE
15
● OpenStack UnderCloud● OpenStack Overcloud control
plane● OpenStack Overcloud compute● OpenShift/Kubernetes master node
VMs● OpenShift routing, registry, and
metrics● OpenShift/Kubernetes minions● Load generators
SYSTEM VERIFICATION TEST SUITE
● Red Hat OpenShift Performance and Scalability team’s upstream test suites● Main tests are
○ cluster-loader○ Networking/synthetic○ Workload Generator○ Reliability/Longevity
● https://github.com/openshift/svt
16
17
CLUSTER-LOADER ARCHITECTURE
Start Parse args & config
ConfigObj
End
False
Create Namespace
True
XExists
?Items < N
False
Create X
Iterate ItemCount
True True
False
X can be:● Quota● Template● Service● User● Pod● RC
CLUSTER LOADER TEST
18
OpenShift (Kubernetes) Object Quantity
Nodes 1,000
Namespaces (projects) 13,000
Pods 52,000
Build Configs 39,000
Templates 78,000
Image Streams 13,000
Deployment Configs and Services 39,000 (incl. 13,000 Replication Controllers)
Secrets 260,000
Routes 39,000
Component # of bugs filed/encountered
Kubernetes 8
Installer 8
Images 5
Docker 5
Containers 3
Storage 2
Ansible 2
Management 1
iptables 1
Deployments 1
Command 1
Builds 1
Auth 1
NOTABLE SCALE ISSUES ENCOUNTERED
19
etcd Disk Utilization API Server CPU
NOTABLE SCALE ISSUES ENCOUNTERED
20
Ansible Installer https://github.com/ansible/ansible/issues/16749
FUTURE OPPORTUNITIES
● Reference architecture available at red.ht/2ibNmvX● Containers and OpenStack: A platform for distributed applications, paper
available at http://red.ht/2hSfIPs● Learn more about Red Hat Summit at redhat.com/summit
21
THANK YOU
22