Elasticsearch &Docker

RafałKuć– Sematext Group,Inc.@kucrafal @sematext sematext.com

Running HighPerformanceFault Tolerant

ElasticsearchClusters OnDocker

Next 20minutes

You Are Probably Familiar WithThat

Development

You Are Probably Familiar WithThat

Development Test

You Are Probably Familiar WithThat

Development Test QA

You Are Probably Familiar WithThat

Development Test QA

Production enviroment

AndProblems That Come WithIt

Resources notutilized

AndProblems That Come WithIt

Resources notutilized

OverprovisonedServers

AndProblems That Come WithIt

Resources notutilized

OverprovisonedServers

≠ ≠

Thesolution

Development Test QA Production

Why Docker?

Light weight

Based onOpenStandards

Secure

Containers vsVirtualMachines

Hardware

Traditional VirtualMachine

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Traditional VirtualMachine

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Traditional VirtualMachine

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Guest OS Guest OS

Traditional VirtualMachine

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Guest OS Guest OS

Libraries Libraries

Traditional VirtualMachine

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Guest OS Guest OS

Libraries Libraries

Application1 Application2

Traditional VirtualMachine

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Guest OS Guest OS

Libraries Libraries

Application1 Application2

Hardware

HostOperatingSystem

Traditional VirtualMachineContainer

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Guest OS Guest OS

Libraries Libraries

Application1 Application2

Hardware

HostOperatingSystem

Docker Engine

Traditional VirtualMachineContainer

Containers vsVirtualMachines

Hardware

HostOperatingSystem

Hypervisor

Guest OS Guest OS

Libraries Libraries

Application1 Application2

Hardware

HostOperatingSystem

Docker Engine

Libraries Libraries

Application1 Application2

Traditional VirtualMachineContainer

Why Elasticsearch?

Distributedbydesign

http://www.dailypets.co.uk/2007/06/17/kittens-rest-at-half-time/

Indices Aggs Admin

Monitor Search

Index

Running Offical ElasticsearchContainer

$docker run-delasticsearch

Running Offical ElasticsearchContainer

$docker run-delasticsearch:latest

Running Offical ElasticsearchContainer

$docker run-delasticsearch:latest$docker run-d--name es1elasticsearch

Running Offical ElasticsearchContainer

$docker run-delasticsearch:latest$docker run-d--name es1elasticsearch$docker run-d--name es1-eES_HEAP_SIZE=1gelasticsearch

Running Offical ElasticsearchContainer

$docker run-delasticsearch:latest$docker run-d--name es1elasticsearch$docker run-d--name es1-eES_HEAP_SIZE=1gelasticsearch$docker run-d--name es1elasticsearch -Dnode.name bbuzz

Container Constraints

$docker run-d-m2G elasticsearch

$docker run-d-m2G--memory-swappiness=0 elasticsearch

http://docs.docker.com/engine/reference/run/

Container Constraints

$docker run-d-m2G elasticsearch

$docker run-d-m2G--memory-swappiness=0 elasticsearch

$docker run-d--cpuset-cpus="1,3"elasticsearch

http://docs.docker.com/engine/reference/run/

Container Constraints

$docker run-d-m2G elasticsearch

$docker run-d-m2G--memory-swappiness=0 elasticsearch

$docker run-d--cpuset-cpus="1,3"elasticsearch

http://docs.docker.com/engine/reference/run/

$docker run-d--cpu-period=50000--cpu-quota=25000elasticsearch

Constraints - GoodPractices

Limitcontainermemory

Constraints - GoodPractices

Limitcontainermemory

Account forI/Ocachewhen givingmemory

Constraints - GoodPractices

Limitcontainermemory

Account forI/Ocachewhen givingmemory

Limitamount ofCPUcores

Constraints - GoodPractices

Limitcontainermemory

Account forI/Ocachewhen givingmemory

Limitamount ofCPUcores

Remember about JVMGC

Creating Optimized Image

Dockerfile:FROM elasticsearchADD ./elasticsearch.yml /usr/share/elasticsearch/config/

Creating Optimized Image

Dockerfile:FROM elasticsearchADD ./elasticsearch.yml /usr/share/elasticsearch/config/

$ docker build -t bbuzz/example .

Creating Optimized Image

Dockerfile:FROM elasticsearchADD ./elasticsearch.yml /usr/share/elasticsearch/config/

$ docker build -t bbuzz/example .

Sending build context toDockerdaemon 5.12kBStep1:FROMelasticsearch--->1e23f30a3667Step2:ADD./elasticsearch.yml /usr/share/elasticsearch/config/--->015f12adfd2aRemoving intermediate container de560c6ae0d1Successfully built 015f12adfd2a

Dealing WithNetwork

$docker run -d -p9200:9200-p9300:9300elasticsearch

Dealing WithNetwork

$docker run -d -p9200:9200-p9300:9300elasticsearch$docker run -d --linkes1 elasticsearch

-Ddiscovery.zen.ping.unicast.hosts=es1

Dealing WithNetwork

$docker run -d -p9200:9200-p9300:9300elasticsearch

Add network.publish_hostwhen buildingown container

$docker run -d --linkes1 elasticsearch-Ddiscovery.zen.ping.unicast.hosts=es1

Dealing WithNetwork

$docker run -d -p9200:9200-p9300:9300elasticsearch

Add network.publish_hostwhen buildingown containerAdd discovery.zen.ping.unicast.hostswhen buildingown

container

$docker run -d --linkes1 elasticsearch-Ddiscovery.zen.ping.unicast.hosts=es1

Network- Good Practices

Separate networkforElasticsearchcluster

Network- Good Practices

Separate networkforElasticsearchcluster

Common hostnames forcontainers$docker run -d -hesnode1elasticsearch

Network- Good Practices

Separate networkforElasticsearchcluster

Common hostnames forcontainers$docker run -d -hesnode1elasticsearch

Exposeonly needed ports

Network- Good Practices

Separate networkforElasticsearchcluster

Common hostnames forcontainers$docker run -d -hesnode1elasticsearch

Exposeonly needed ports

Elasticsearchdata &client nodes pointtomasters only

Dealing WithStorage

Bydefault in/usr/share/elasticsearch/data

Dealing WithStorage

Bydefault in/usr/share/elasticsearch/data

Bydefault notpersisted

Dealing WithStorage

Bydefault in/usr/share/elasticsearch/data

Bydefault notpersisted

$docker run -d-v /opt/elasticsearch/data:/usr/share/elasticsearch/dataelasticsearch

Dealing WithStorage

$docker run -d-v /opt/elasticsearch/data:/usr/share/elasticsearch/dataelasticsearch

Bydefault in/usr/share/elasticsearch/data

Bydefault notpersisted

Use dataonly Dockervolumes

Permissions

DataOnly Docker Volumes

Bypasses UnionFileSystem

DataOnly Docker Volumes

Bypasses UnionFileSystem

Can beshared between containers

DataOnly Docker Volumes

Bypasses UnionFileSystem

Can beshared between containers

Datavolumes persist if thecontainer itself is deleted

DataOnly Docker Volumes

Bypasses UnionFileSystem

Can beshared between containers

Datavolumes persist if thecontainer itself is deleted

$docker create -v/mnt/es/data:/usr/share/elasticsearch/data--name esdata elasticsearch

Permissions

DataOnly Docker Volumes

Bypasses UnionFileSystem

Can beshared between containers

Datavolumes persist if thecontainer itself is deleted

$docker create -v/mnt/es/data:/usr/share/elasticsearch/data--name esdata elasticsearch

$docker run --volumes-fromesdata elasticsearch

HighlyAvailable Cluster

Masteronly

Masteronly

Masteronly

Dataonly

Dataonly

Dataonly

Dataonly

Dataonly

Dataonly

Clientonly

Clientonly

HighlyAvailable Cluster

Masteronly

Masteronly

Masteronly

Dataonly

Dataonly

Dataonly

Dataonly

Dataonly

Dataonly

Clientonly

Clientonly

minimum_master_nodes=N/2+1

MasterNodes &Docker

$docker run-d elasticsearch-Dnode.master=true-Dnode.data=false-Dnode.client=false

ClientNodes &Docker

$docker run-d elasticsearch-Dnode.master=false-Dnode.data=false-Dnode.client=true

DataNodes &Docker

$docker run-d elasticsearch-Dnode.master=false-Dnode.data=true-Dnode.client=false

Multiple Tiers

node.tag=hot node.tag=cold node.tag=cold

Multiple Tiers

$docker run-d elasticsearch -Dnode.tag=hot

Multiple Tiers

curl -XPUT 'localhost:9200/data_2016-06-05' -d '{ "settings": { "index.routing.allocation.include.tag" : "hot"

}}'

Multiple Tiers

node.tag=hot node.tag=cold node.tag=cold

data_2016-06-05

data_2016-06-05

Multiple Tiers

node.tag=hot node.tag=cold node.tag=cold

data_2016-06-05

data_2016-06-05

TopMetrics – Health &Shards

https://sematext.com/spm/

TopMetrics - CPU

https://sematext.com/spm/

TopMetrics – MemoryUsage

https://sematext.com/spm/

TopMetrics – I/OUsage

https://sematext.com/spm/

TopMetrics – JVMHeap

https://sematext.com/spm/

TopMetrics – Garbage Collector

https://sematext.com/spm/

TopMetrics – Request Rate &Latency

https://sematext.com/spm/

TopMetrics - Caches

https://sematext.com/spm/

TopMetrics – Indexing

https://sematext.com/spm/

TopMetrics – Refresh Time

https://sematext.com/spm/

Top10Metrics – Merge Time

https://sematext.com/spm/

Short summary

http://www.soothetube.com/2013/12/29/thats-all-folks/

WeAre Hiring !

Dig Search ?Dig Analytics?Dig BigData?Dig Performance?Dig Logging ?Dig working withandinopen– source ?We’re hiring world – wide !

http://sematext.com/about/jobs.html

RafałKuć@kucrafalrafal.kuc@sematext.com

Sematext@sematexthttp://sematext.comhttp://blog.sematext.com

Thank You !