Date post: | 13-Feb-2017 |
Category: |
Documents |
Upload: | dangkhuong |
View: | 230 times |
Download: | 2 times |
Using AWS to Build a Large Scale
Dockerized Microservices ArchitectureDr. Oliver Wahlen
moovel Group GmbH
Frankfurt, 30. Juni 2016
The moovel Group GmbH
Our vision is an ecosystem that
• simplifies mobility now
• solves the mobility challenges of the future
Mobility as a Service
Urban areas will become more crowded:
• Today 54% of people live in cities In 2050 there will be 66%
• More people on less space
• Driving and parking will become more regulated
• High demand for new transportation concepts
Autonomous driving will become reality:
• Taxi and car sharing business will change completely
• Cost efficiency similar to public transport but faster
• Owning a car becomes less relevant
The future will be Mobility as a Service
Entwicklung 3: Autonomes Fahren
Wir verändern die Art,
wie wir uns fortbewegen.
We are building apps, widgets and services that integrate
existing transportation providers
Benefits for the user:
• Only one registration: no need to remember passwords
or provide payment data several times
• Transparency on mobility alternatives
• Intermodal routing
Benefits for the partners:
• Get new users from other mobility providers
• Get new users from different geographic regions
moovel Technology
Entwicklung 3: Autonomes Fahren
Wir verändern die Art,
wie wir uns fortbewegen.
• Scalability up to millions of users
• Future prove: Easy to modify components and update technologies
• Usable on global scale
• Minimal operational effort
• Teams can develop independently
Design Goals of Our System Architecture
Entwicklung 3: Autonomes Fahren
Wir verändern die Art,
wie wir uns fortbewegen.
Avoid Vendor Lock-In vs. Reduced Operational Effort
Services currently
used by moovel
Entwicklung 3: Autonomes FahrenWhat and Why Docker?
Docker Containers
• Provide application(s) with a virtual operating system
• Isolate applications from the host OS and other docker containers
(e.g. Files, Network)
• Are self contained run on any machine without other installs
• Are simple to build images reference a base image
Docker Registry
• Stores docker images produced by the build system
• Provides access control and maintenance of tags
• Dockerhub (Docker Inc.) vs. Amazon EC2 Container Registry
Entwicklung 3: Autonomes FahrenWhat and Why EC2 Container Service (ECS)?
ECS is a highly scalable, fast, container management service that makes
it easy to run, stop, and manage Docker containers on a cluster of
Amazon EC2 instances.
– AWS documentation
Technology similar to Kubernetes on Google Container Engine
but tailored for AWS and fully managed
Terminology
• ECS cluster: Group of EC2 instances that allow docker execution
• ECS container instance: EC2 instance with ECS container agent
• ECS task: The running docker container on an instance
• ECS task definition: docker image, CPU, mem, port requirements, …
• ECS service: number of parallel tasks, scalability, health checks, …
Entwicklung 3: Autonomes FahrenWhat and Why Microservices?
http://martinfowler.com/articles/microservices.html
• Services independently deployable
Continuous Deployment
• Scalable on service level
Specific to service needs:
CPU, memory
• Services have firm boundary
allows for:
different languages per service
different teams per service group
evolution of logic and technology
ECS Cluster
EC2 Instance
Docker Task
= ECS Task
Web Service
Entwicklung 3: Autonomes FahrenConway’s Law And Microservices: A Good Fit
Organizations which design
systems [...] are constrained to
produce designs which are copies
of the communication structures
of these organizations
– Melvin Conway 1967
Product Owner
App-Developers
Backend-Devs
Search
Squad
Book
SquadUser
Squad
Pay
Squad
moovel product dev is organized in squads (feature teams):• with a long term business vision• ability to build and modularize mobile apps / componets• build their own set of microservices• choose the technology that best fits their needs
Entwicklung 3: Autonomes FahrenContinuous Integration with Docker and AWS
Developer
code push to
SaaS Build SystemVersion Control
triggers dockerfile.tgz
S3 Bucket
Build Path
SaaS Build System Dockerhub
Deployment triggers:
develop branch: automatic
master branch: manually
Deployment Path
docker image
push to
start update
CloudformationElastic Container
Services: dev/prod
update docker
Entwicklung 3: Autonomes Fahrenmoovel System Architecture
moovel VPC
Client App
External ELB• TLS termination
Kong API Gateway• reverse proxying
• LUA plugins
Consul• service discovery
• client load balancing
• service configuration
synchronous REST call
ECS microservice
SNS
Topic
SQS
Queue
asynchronously
publish changes
logging Kinesis ELK Stack
Entwicklung 3: Autonomes FahrenLimitations of Existing Solution
• Difficult to realize dynamic port allocation
Currently each service has a fixed port
• Two levels of scaling introduce additional complexity:
Cluster nodes = No. of EC2 instances
ECS Tasks = No. of running Docker containers for a service
had to introduced a custom metric taskless instance count
• Service discovery with Consul:
Automates node and service registry
Provides hierarchical key value store with ACL for configuration
Consul is a very critical component
Must be operated and maintained
• Kong API gateway:
Extremely flexible and powerful nginx based reverse proxy
Requires cassandra DB
Must be operated and maintained
• Elasticsearch Logstash Kibana (ELK):
Do not use it for long term data storage!
Entwicklung 3: Autonomes FahrenCheck-up of Design Goals
Scalability
moovel runs 80 services on 21 ECS clusters (dev+prod)
Future prove
Exchanging microservice technology is easy
We have microservices in Java, Node.js, Python, Kotlin, …
Global Scale
The world is currently operated from eu-west-1
Pub/Sub is good starting point for multi region deployment
Minimal Operational Effort
Technology is operated by 10 squads
Currently no DevOps team needed: squads build it and run
Deployments are easy, frequently done and without downtime
Teams can develop independently
Technology is operated by 10 squads
Microservices, REST and Pub/Sub are a good basis
UI decomposition in app is difficult: modularization ongoing
Thank You