Living with microservices at Pipedrive

transcript

Living with microservices@Pipedrive

Renno ReinumA developer’s best friend

KerberosThe Guard of Underworld

Deploy or not to deploy

That is the question

● Pipedrive helps small businesses control the complex selling process

● Founded in 2010● 30,000 paying customers worldwide● 170+ employees● Office in Tallinn and Tartu

New York, NY

● 20,000+ simultaneous online users

● 800+ API req/sec

● 400,000+ incoming emails per day

● Started with Node.js based microservices, reactive

architecture in 2012

● In total, 500+ VMs/hosts/instances

● More than 300 deploys per month

Pipedrive helps small businesses control the complex selling process

Evolution of the application

The birth of monolith: Only kanban board

New features added:Monolith growing

Bloated monolith:Application slowing down

Once upon a time

● We need to keep the pace of delivering services!

● Teams need to be independent of each other● How can we add new features daily basis and

still maintain it?● Lets split it into microservices

Introduction to Microservice

Developers should write programs that can communicate easily with other programs.

Break down projects into small, simple programs rather than overly complex monolithic programs.

Eric Raymond - Unix Philosophy

Monolith vs Microservices

● Microservices systems are more flexible, loosely-coupled and scalable● Easier to develop and easier to change● More tolerant of failure. In case of one mircoservice fails, others should be

operational

http://www.randomant.net/the-cloud-is-a-paradigm-shift-in-software-dev/

And the new era begins

Deployment of new microservice

From idea to production - minimum 2 WEEKS

Characteristics of microservices (by Martin Fowler)

● Componentization via services

● Organized around business capabilities

● Products not projects

● Smart endpoints & dumb pipes

● Decentralized governance● Design for failure

● Infrastructure automation

Virtual Machines vs Containers (Docker)

http://www.docker.com/what-docker

Visibility

Management

Interface

Visibility

Management

Interface

How not to create microservices or any other automation flows

● CI job hell● Discovery● Configurations hell

Mindshift of Developers

● Building● Testing● Discovery● Monitoring● Logging● Alerting● Dashboards

Agile Culturein Pipedrive

Agile and lean means small and fast and independent and in Pipedrive we operate by small cross-functional teams which deliver regularly. Those teams win small battles every day but their operations are supported by different types of other troops.

Devops Culture

in Pipedrive

Mindshift of Developers -Now I master the Microservice concept

● Dockerfiles● Docker-compose● CI builds● Tests● Service discovery● From idea to production

CI processes with Docker● Provisioning of build

environment● Pulling Docker images● Docker builds

xkcd.com

CI processes with Docker● Whole CI process can take a 1 - 3

minutes

xkcd.com

CI processes with DockerFeedback from Developers:

● It’s still too sl.oOW

xkcd.com

CI processes with DockerFeedback from Developers:

● It’s still too sl.oOW● Make it slower, I can’t grab a

coffee

xkcd.com

Service developer wants to run

Services we require to run on the server

Challenges to be resolved

● Multi-datacenter/regions● Autoscaling of containers● Rollbacks● Secrets management● Development environment

Beware of the Gold Fever

“To make error is human. To propagate error to all server in automatic way is #devops.”

Final Thoughts

● Define the problem you need to solve● The right answer > 1

Final Thoughts

● Define the problem you need to solve.● The right answer > 1

Thank you!

Feedback is highly encouraged :)

@rreinurm

Living with microservices at Pipedrive

Engineering