Post on 06-Jan-2017
transcript
Microservices with Docker Swarm and Consul
Agenda
- Something about Microservices
- Implement Microservices with Docker Swarm and Consul
“You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile”(Martin Fowler)
Some SuggestionsPrograming: Python, NodeJS, Golang*
Orchestration: Docker, Docker Swarm, Consul, Consul-Template, Registrator, CoreOS*
Databases & Caches: PosgreSQL, Cassanda, ElasticSearch, Redis
Load Balancer & API Gateway: Nginx
Queues: Kafka
CI/CD: Jenkins, Ansible
Monitors: Grafana + InfluxDB + (CollectD, cAdvisor)
Logs: Kibana + ElasticSearch + FluentD
Data Center: AWS EC2, AWS S3, AWS Float IP, Cloudfront
CDN & DNS & DDOS Protection: Cloudflare
Documents: Swagger, Markdown, mkdocs
Backend System Suggestions
An Example
Node Gateway will be used a API Gateway and Load Balancer
Node Agent One, Node Agent Two will be used to run services that are able to scale.
Docker & Docker SwarmDocker is a platform for developers and sysadmins to develop, ship, and run applications. Docker lets you quickly assemble applications from components and eliminates the friction that can come when shipping code. Docker lets you get your code tested and deployed into production as fast as possible.
Docker Swarm is native clustering for Docker. It allows you create and access to a pool of Docker hosts using the full suite of Docker tools. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts
#install
$ docker pull swarm
#run joiner
$ docker run -d swarm join \ --addr=172.20.20.11:2375 \ token://acdb9dfa3ea6da0b0cfb2c819385fcd3
#run manager
$ docker run -d -p 12375:2375 \swarm manage \token://acdb9dfa3ea6da0b0cfb2c819385fcd3
Consul and Consul-TemplateConsul
- Service Discovery
- Key/Value Storage
- Failure Detection
- Multiple Datacenter
Consul-Template
- Listen/Query the updates from Consul
- Updates Configuration File by the templates provided
- Executes commands
#run a consul agent$ consul agent -data-dir /tmp/consul -node=agent-one \ -bind=172.20.20.11 -client=0.0.0.0 \
-config-dir /etc/consul.d \ -retry-join 172.20.20.10
#run consul-templateexec consul-template \ -consul 127.0.0.1:8500 \ -template "/build/gateway/consul-template/nginx.ctmpl:/etc/nginx/sites-available/default:service nginx reload" \ -retry 30s
RegistratorRegistrator automatically registers and deregisters services for any Docker container by inspecting containers as they come online. Registrator supports pluggable service registries, which currently includes Consul, etcd and SkyDNS 2.
#run registrator
$ docker run -d \ --name=registrator --net=host \ --volume=/var/run/docker.sock:/tmp/docker.sock \ gliderlabs/registrator:latest \ consul://localhost:8500
A Scenario
At the beginning, Service #1 only runs in Node 2.
After that Service #1 scales to Node 2
Via Docker API, Registrator will know the appearance of Service #1 on Node 2.
Registrator will register Service #1 with Consul.
In the meanwhile, Consul-Template in listening Consul. So Consul-Template will know the new service is up in Node 2. Finally, Consul-Template will trigger a reload command to Nginx add the new service into load balancer.
DEMOhttps://github.com/thanhson1085/microservices-swarm-consul
MICROSERVICES
DEVOPS
References
http://martinfowler.com
http://microservices.io/patterns/
https://www.sequoiacap.com/article/build-us-microservices/
https://sonnguyen.ws/microservices-with-docker-swarm-and-consul/