Microservice

The Evolutionary Architecture

Software Engineering II

Sharif University of Technology

MohammadAmin Fazli

Evolutionary Architecture

Topics

An Evolutionary Vision for the Architecture

Zoning

A Principal Approach

Governance Through Code

Technical Debt

Exception Handling

Governance from the Center

Building a Team

Reading:

Building Microservices-Sam Newman-Chapter II

2

Evolutionary Architecture

Inaccurate Comparisons

We are not real architectures in software engineering

Architects and engineers have a rigor and discipline we could only dream of, and their importance in society is well understood

We plainly don’t what we are doing.

The number of times buildings and bridges fall down is much less than the number of times our programs will crash.

Our software isn’t constrained by the same physical rules that real architects or engineers have to deal with, and what we create is designed to flex and adapt and evolve with user requirements.

Perhaps the term architect has done the most harm

The idea of someone who draws up detailed plans for others to interpret, and expects this to be carried out.

In our industry, this view of the architect leads to some terrible practices: many diagrams created with a view of the perfect system, utterly devoid of any understanding as to how hard it will be to implement, or whether or not it will actually work

3

Evolutionary Architecture

An Evolutionary Vision for the Architect

Our requirements shift more rapidly than they do for people who design and build buildings.

The things we create are not fixed points in time: our software will continue to evolve as the way it is used changes.

Our architects need to shift their thinking away from creating the perfect end product, and instead focus on helping create a framework in which the right systems can emerge, and continue to grow as we learn more.

We are more as town planner than architects

Planners’ roles:

Look at a multitude of sources of information

Optimize the layout of a city to best suit the needs of the citizens

Zonning the city; does not say “build this specific building here”

Town planners does not worry much about what happens inside a zone, he spends time working on how things move from one zone to other.

4

Evolutionary Architecture

An Evolutionary Vision for the Architect

We are more as town planner than architects

City/Software changes over time; It shifts and evolves as its occupants use it in different ways, or as external forces shape it. The town planner/architect does his best to anticipate these changes, but accepts that trying to exert direct control over all aspects of what happens is futile.

City/Software must be habitable for

City: Different citizens

Software: Users, Developers, Operation people

A town planner/Architect needs to know when his plan isn’t being followed.

As he is less prescriptive, the number of times he needs to get involved tocorrect direction should be minimal, but if someone decides to build a sewage plant in a residential area, he needs to be able to shut it down.

Architects as town planners need to set direction in broad strokes, and only get involved in being highly specific about implementation detail in limited cases

5

Evolutionary Architecture

An Evolutionary Vision for the Architect

6

Evolutionary Architecture

Zoning

What are zones?

Service boundaries?

Coarse grained groups of services?

As architects, we need to worry much less about what happens inside the zone than what happens between the zones

How they talk with each other

Ensuring that we can properly monitor the overall health of our system

Within each service, architect may be OK with the team who owns that zone picking a different technology stack or data store.

It may have some drawbacks

Hiring people

Hard to scale

7

Evolutionary Architecture

Zoning

Architect may establish some standards on the zones

Netflix standard on choosing Cassandra as data storage

Integration technology: REST, Java RMI, Protocol buffers (ex. Apache

Thrift)

Coding standards: Pair programming, QSD best practices, ….

A very important best practice: Architect should join teams

for a short period as one member of the pair.

Ideally, architect should work on normal stories, to really understand

what normal work is like.

8

Evolutionary Architecture

A Principal Approach

Making decisions in system

design is all about trade-offs,

and microservice architectures

give us lots of trade-offs to

make.

The principal approach

Strategic goals

Principles

Practices

Combining principles and

practices

9

The Principal Approach to the Great Dagon Pagoda at Rangoon

Joseph Moore

Evolutionary Architecture

Strategic goals

The role of the architect is not to define strategic goals

Strategic goals should speak to where your company is going,

and how it sees itself as best making its customers happy.

Expand into Southeast Asia to unlock new markets

Let the customer achieve as much as possible using selfservice

The key is that this is where your organization is headed, so

you need to make sure the technology is aligned to it.

This is more a business job

10

Evolutionary Architecture

Principles

Principles are rules you have made in order to align what you are doing to some larger goal, and will sometimes change.

Strategic goal: to decrease the time to market for new features Principle: delivery teams have full control over the lifecycle of their software to ship whenever they are ready

Strategic goal: moving to aggressively grow its offering in othercountries Principle: the entire system must be portable to allow for it to be deployed locally in order to respect sovereignty of data

About 10 principles is good.

Example: Heroku’s 12 Factors are a set of design principles structured around the goal of helping you create applications that work well on the Heroku platform

Some of the principles are actually constraint based on behaviors your application needs to exhibit in order to work on Heroku.

A constraint is really something that is very hard (or virtually impossible) to change, whereas principles are things we decide to choose.

11

Evolutionary Architecture

Practices

The practices are how we ensure our principles are being

carried out. hey are a set of detailed, practical guidance for

performing tasks.

Often technology specific

Low level enough that any developer can understand them

Will often change more often than principles

Examples:

Coding guidelines

All log data needs to be captured centrally

HTTP/REST is the standard integration style.

12

Evolutionary Architecture

Practices

As with principles, sometimes practices reflect constraints in

your organization.

If you support only CentOS, this will need to be reflected in your

practices.

Practices should underpin our principles.

Example Principle: delivery teams control the full lifecycle of their

systems

Example Practice: All services are deployed into isolated AWS

accounts

13

Evolutionary Architecture

Combining Principles and Practices

One person’s principles are another’s practices.

Example: You might decide to call the use of HTTP/REST a principle rather than a practice

Is mixing principles and practices a good idea?

The key point is that there is value in having overarching ideas that guide how the system evolves, and in having enough detail so that people know how to implement those ideas.

For a small enough group, perhaps a single team, combining principlesand practices might be OK.

For larger organizations, where the technology and working practices may differ, you may want a different set of practices in different places, as long as they both map to a common set of principles

14

Evolutionary Architecture

Example

15

Evolutionary Architecture

The Required Standard

The System needs to be a cohesive system made of many

small parts with autonomous lifecycles but all coming together

We should not allow too much divergence

We need a tradeoff between optimization and autonomy

Standardization issues:

Monitoring: It is essential that we are able to draw up coherent,

cross-service views of our system health.

Mechanism: Push & Poll

Technology (Graphite for metrics, Nagios for system health)

Interfaces: Picking a small number of defined interface technologies

helps integrate new consumers.

Having one standard is a good number. Two isn’t too bad, either. More is bad.

16

Evolutionary Architecture

The Required Standard

Standardization issues:

Architectural Safety: We have to ensure that our services shield

themselves from unhealthy, down-stream calls.

The more services we have that do not properly handle the potential failure

of downstream calls, the more fragile our systems will be.

This means you will probably want to mandate as a minimum that each

downstream service gets its own connection pool, and you may even go as far

as to say that each also uses a circuit breaker.

Response codes must be standard: For HTTP, one service decides to send

back 2XX codes for errors, or confuses 4XX codes with 5XX codes, then

these safety measures can fall apart.

17

Evolutionary Architecture

Circuit Breaker Pattern

To wrap a protected function

call in a circuit breaker

object, which monitors for

failures. Once the failures

reach a certain threshold, the

circuit breaker trips, and all

further calls to the circuit

breaker return with an error,

18

Evolutionary Architecture

Connection Pool

A connection

pool is a cache

of database

connections

maintained so

that the

connections

can be reused

when future

requests to the

database are

required.

19

Evolutionary Architecture

Governance Through Code

Making sure people are following our designed guidelines is

very hard

It is placing a burden on developers to implement all these standard

things you expect each service to do.

Two techniques come here:

Exemplars: If you have a set of standards or best practices you would

like to encourage, then having exemplars that you can point people

to is useful. The idea is that people can’t go far wrong just by

imitating some of the better parts of your system.

These should be real-world services you have that get things right, rather than

isolated services that are just implemented to be perfect examples.

Tailored Service Template: The developers had most of the code in

place to implement the core attributes that each service needs.

Next slide

20

Evolutionary Architecture

Tailored Service Template

Some technologies make building them easier

Dropwizard and Karyon are two open source, JVM-based microcontainers.

They work in similar ways, pulling together a set of libraries to provide features like health checking, serving HTTP, or exposing metrics. So you have a service complete with an embedded servlet container that can be launched from the command line.

Example: Enabling circuit breaker pattern

Integrate a circuit breaker library like Hystrix

A practice that all your metrics need to be sent to a central Graphite server, so perhaps pull in an open source library like Dropwizard’s Metrics and configure it so that response times and error rates are pushed automatically to a known location.

Multiple technology stacks needs multiple matching service template for each.

21

Evolutionary Architecture

Tailored Service Template

Case: Sidecar Services

Netflix is concerned with fault tolerance

A large amount of work has been done to ensure that there are

client libraries on the JVM to provide teams with the tools they need to keep their services well behaved

Anyone introducing a new technology stack would mean having to

reproduce all this effort.

The risk of a service getting newly implemented fault tolerance

wrong is high if it could impact more of the system.

22

Evolutionary Architecture

Tailored Service Template

Case: Netflix Sidecar Services

Netflix is concerned with fault tolerance

A large amount of work has been done to ensure that there are

client libraries on the JVM to provide teams with the tools they need to keep their services well behaved

Anyone introducing a new technology stack would mean having to

reproduce all this effort. The risk of a service getting newly

implemented fault tolerance wrong is high if it could impact more of

the system.

Netflix mitigates this by using sidecar services, which communicate

locally with a JVM that is using the appropriate libraries.

23

Evolutionary Architecture

Tailored Service Template

Creating the service template must not become the job of a

central tools or architecture team who dictates how things

should be done, albeit via code. It should be a collective

activity (open source approach)

Ideally, its use should be purely optional, but if you are going to

be more forceful in its adoption you need to understand that

ease of use for the developers has to be a prime guiding force.

In our desire to create reusable code, we can introduce

sources of coupling between services.

Low coupling, Low reuse: Copying service code

High coupling, High reuse: Shared binary dependency

24

Evolutionary Architecture

Technical Debt

Often, we need to make a choice to cut a few corners to get some urgent features out.

If we deviate from our technical vision, it might have a

short-term benefit but a long-term cost.

When we accrue technical debt, just like debt in the

real world it has an ongoing cost, and is something we

want to pay down.

Sometimes debt is caused by changing the vision.

The architect’s job is to look at the bigger picture, and

understand this balance. Having some view as to the

level of debt, and where to get involved, is important.

Sometimes teams themselves must track and pay down their

debt

Sometimes a complete involvement is required

25

Evolutionary Architecture

Exception Handling

Sometimes we make a decision that is just an exception to the

rule.

In these cases, it might be worth capturing such a decision in

a log somewhere for future reference.

If enough exceptions are found, it may eventually make sense to

change the principle or practice to reflect a new understanding

of the world.

26

Evolutionary Architecture

Governance & Leading from the Center

Governance ensures that enterprise objectives are achieved by

evaluating stakeholder needs, conditions and options; setting

direction through prioritisation and decision making; and monitoring

performance, compliance and progress against agreed-on direction and objectives (COBIT 5)

Architecture’s job is technical governance (A lot of things to do)

Normally, governance is a group activity.

It could be an informal chat with a small enough team or a more structured

regular meeting with formal group membership for a larger scope.

Principles will be changed here as required

This group needs to be led by a technologist, and to consist predominantly

of people who are executing the work being governed. This group should

also be responsible for tracking and managing technical risks.

27

Evolutionary Architecture

Governance & Leading from the Center

Recommendation:

Having the architect chair the group, but having the bulk of the group

drawn from the technologists of each delivery team.

The architect is responsible for making sure the group works, but

the group as a whole is responsible for governance.

Architect must not force his decision, but he should convince the

group.

28

Evolutionary Architecture

Building a Team

Great software comes from great people

If you worry only about the technology side of the equation, you’re missing way more than half of the picture.

Helping the people around you on their own career growth can take many forms, but there is one aspect where microservices can help

With larger, monolithic systems, there are fewer opportunities for people to step up and own something.

With microservices, on the other hand, we have multiple autonomous codebases that will have their own independent lifecycles.

Helping people step up by having them take ownership of individual services before accepting more responsibility can be a great way to help them achieve their own career goals, and at the same time lightens the load on whoever is in charge!

29