Testing in the hundred microservices world. When the pyramid becomes an hourglass

Isa Vilacides Director Engineering at CloudBees

#EuroSTARConf

Index1. Monolith vs microservice

2. Traditional model and it’s challenges in distributed systems

3. Fast, robust and reliable integration testing is possible

4. Reshaping the pyramid and being the first ones to know if something went wrong in production

5. A bit more about safe rollouts

Monolith vs Microservice architecture

Traditional testing pyramid in monoliths and its challenges in the micro services world

Testing pyramid

End to end tests in the monolith- Slow

- Unreliable

- Difficult to maintain

- Difficult to debug

- Test real integrations

End to end tests in distributed systems- Slow

- Even more unreliable

- Difficult to maintain

- Difficult to debug

- Test real integrations

Unit and integration tests in the monolith

- Fast

- Reliable

- Easy to maintain and debug

- Based on assumptions

Integration tests in distributed systems- Require to spin up or mock 1+ services

- Not that fast

- Not that reliable

- Easy to maintain and debug

- Based on assumptions

Based on assumptions

What if what you are mocking changes?

Based on assumptions

It relies on the

developer watching for

those changes

What if you could get the best of the two worlds?

Fast, robust and reliable integration testing is possible

Consumer driven contract testing

- Fast feedback in isolation

- Early detection in both sides

- Enables communication

Reshaping the pyramid

Reshaping the pyramid

You want to go fast without breaking things, but things break so you want to be the first one to know that things went wrong in production

Reshaping the pyramid

- Giving importance to post deployment verification

- Deployment health checks

- Monitoring

Reshaping the pyramid

- Understand how your apps normally behave

- % of 500 responses per second

- Establish thresholds

- Measure and monitor

- Raise alerts

Reshaping the pyramid

Testing pyramidPDV

monitoring

Reshaping the pyramid

By when you measure and react customers are already affected

A bit more about safe rollouts

Dogfooding, feature flag development and staged rollouts

Feature flags

Canary rollouts

Dogfooding

Summary

• Focus on integrations between consumers and providers

• If something goes wrong in production you want to be the first one to know about it

• If something goes wrong in production you want it to affect the least customers possible

Thank You! Questions?

We are hiring @CloudBees

Twitter @vilacides

#EuroSTARConf