Page 1 © Hortonworks Inc. 2014
OpenShift scheduling docker containers in YARN via Kubernetes
Ali BajwaPartner Solutions EngineerOct 2014
Page 2 © Hortonworks Inc. 2014
Static website Web frontend User DB Queue Analytics DB
Development VM QA server Public Cloud Contributor’s laptop
Docker: a shipping container system for code M
ultip
licity
of S
tack
sM
ultip
licity
of
hard
war
e en
viro
nmen
ts
Production Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can I migrate
smoothly and quickly
…that can be manipulated using standard operations and run consistently on virtually any hardware platform
An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
Page 3 © Hortonworks Inc. 2014
Why are Docker containers lightweight?
HDP 2.1Hortonworks Data Platform
I/O performance comparable to Bare Metal
Page 4 © Hortonworks Inc. 2014
Kubernetes – container orchestrator
• Service for container cluster management• Allows deploying and managing applications running
on multiple hosts using docker• Open sourced by Google• Supports GCE, CoreOS, Azure, vSphere• Used to manage Docker containers as a default
implementation• Master – maintain state of Kubernetes Server runtime
• Scheduler, API server, registries, storage• Minions – represent the host were containers created
• Kubelet – manage pod and container lifecycle
Page 5 © Hortonworks Inc. 2014
OpenShift
• Red Hat’s platform as a service forapplications in the cloud that supportsboth public and private cloud• Provides high level abstraction for applications on top
of containers allowing easy scaling, service discovery, and deployment
• Enable Docker image authors to easily deliver reusable application components, including highly available databases, monitoring and log aggregation tools, service discovery platforms, and prepackaged web applications
• Allow developers to deeply customize their runtime environments while preserving operational support at scale for those applications
Page 6 © Hortonworks Inc. 20146
Kubernetes/YARN Integration
YARN Node Manager
YARN Node Manager
Kubernetes AppMaster
Page 7 © Hortonworks Inc. 2014
Demo: OpenShift scheduling docker container in YARN
Running in docker
Page 8 © Hortonworks Inc. 2014
Demo: 1. Launch Docker container on YARN via OpenShift
#openshift kube create pods –c demo.json
#cat demo.json
Page 9 © Hortonworks Inc. 2014
Demo: 2. Enter into container and confirm web server started
#openshift kube create pods –c demo.json
Page 10 © Hortonworks Inc. 2014
Demo: 3. Webapp on Openshift interacting with Hadoop
Page 11 © Hortonworks Inc. 2014
Demo: 4. Launch other workloads on YARN
YARN allows Big Data and other workloads to run side by side on same cluster
Page 12 © Hortonworks Inc. 2014
Questions?