+ All Categories
Home > Documents > 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql...

7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql...

Date post: 20-Jul-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
30
طᶪᗑᕶጱ ۸ොໜറᔱ ሴỨ (Tad)
Transcript
Page 1: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

(Tad)

Page 2: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Page 3: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Why Docker?

Page 4: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB
Page 5: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

• GlusterFS, Ceph FS / RDB

• …

Page 6: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Benefits From Docker

• Deployment

• Update

• Universal Method among various OS

Page 7: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Why Use Docker

Page 8: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Deployment

Page 9: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Normal Deployment

• Uniq Method For various DB

• Dependance Problem

• OS Support (CoreOS, CentOS, Debian, etc.)

• Standard Tuning Method

Page 10: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

• Ansible, Fleet

• Server Resource Pool

• Deployment Template (Systemd-unit-file)

How

Page 11: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

• etcd

• zookeeper

• mongodb

• elastic search

• mysql

• hadoop

• ceph

• glusterFS

• kafka

• nsq

Support DB

Page 12: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

ExampleDescription=mongo-bds-uba-rs After=docker.service Requires=docker.service [Service] TimeoutStartSec=0 Restart=always ExecStartPre=-/usr/bin/docker rm mongo-bds-uba-rs ExecStart=/usr/bin/docker run \\ --name mongo-bds-uba-rs \\ -p 1301:27017 \\ -v /hdd1/bds-uba-rs:/data/db \\ dhub.yunpro.cn/library/mongo:3 \\ --storageEngine wiredTiger \\ --replSet bds-uba-rs

ExecStop=/usr/bin/docker kill mongo-bds-uba-rs [Install] WantedBy=multi-user.target

Page 13: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Upgrade

Page 14: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Normal Upgrade

• Rollback Solution

• Dependance Confliction

• Complicated Documents

Page 15: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

But• Small Version

• usually can upgrade directly

• Big Version

• re-import in required OR making compatible scripts to upgrade

• like: ceph, mongo engine switch

Page 16: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

ExampleDescription=mongo-bds-uba-rs After=docker.service Requires=docker.service [Service] TimeoutStartSec=0 Restart=always ExecStartPre=-/usr/bin/docker rm mongo-bds-uba-rs ExecStart=/usr/bin/docker run \\ --name mongo-bds-uba-rs \\ -p 1301:27017 \\ -v /hdd1/bds-uba-rs:/data/db \\ PRIVATE-REGISTRY/library/mongo:3.2 \\ --storageEngine wiredTiger \\ --replSet bds-uba-rs

ExecStop=/usr/bin/docker kill mongo-bds-uba-rs [Install] WantedBy=multi-user.target

Page 17: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Scaling

Page 18: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Example[Unit] Description=mongo-shard After=docker.service Requires=docker.service [Service] TimeoutStartSec=0 Restart=always ExecStartPre=-/usr/bin/docker rm mongo-shard ExecStart=/usr/bin/docker run \

--name mongo-shard \ -p 1201:27018 \ -v /data/shard:/data/db \ PRIVATE-REGISTRY/library/mongo:3 \ --storageEngine wiredTiger \ --shardsvr

ExecStop=-/usr/bin/docker kill mongo-shard [Install] WantedBy=multi-user.target

Page 19: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Support DB• Redis Cluster

• Mongo Shard

• ElasticSearch

• Hadoop

• NSQ

• Kafka

Page 20: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Auto Failover backup

Page 21: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Support DB• etcd failover

• zookeeper failover

• mongoDB replication

• redis master-slave

• mysql master-slave

Page 22: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Example

• Mongo Replication HDD broken

• Select Available Server From Servers Pool

• Start Replication Docker Instance

• Mongo Client Remove Old One and Add New One

Page 23: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Auto Balance

Page 24: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Example

• Mongos

• Using DNS to Round-Robin

• Increase Mongos to handle more connections and traffic

Page 25: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Benchmark

Page 26: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Why Docker

• Easy To deploy in temporary servers

• More Connections

• More Operations to generate

• Build Once and run anywhere

Page 27: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Benchmark

• More Connections and operations to DB

Page 28: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Problems

Page 29: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Problems• MongoDB no Master

• MongoDB Connections Over 20,000

• MongoDB Tuning

• Ceph RDB Volume Support Problem

Page 30: 7 ª Ñ v 1 ; ø Ü Ü 1 1 … · •etcd • zookeeper • mongodb • elastic search • mysql • hadoop • ceph • glusterFS • kafka • nsq Support DB

Thank You

We Are Hiring!

http://www.goyoo.com


Recommended