Moderná orchestrácia microslužieb - DDD community...Introduction 2. Use-Case Description 3....

transcript

Moderná orchestrácia microslužieb

stanislav.ondac@[ferratum|gmail].com

linkedin.com/in/stanislav-ondac

@StanislavOndac

FE-Classification: Confidential\Anyone

1. Introduction

2. Use-Case Description

3. Choreography scenarios

4. Orchestration scenarios

5. Orchestration Live demo

6. Lessons learned

14 years in industry

Roles:

– Current: • Software Architect, • Team Lead, • Deputy Chief Architect

– Past:• Developer : C++, J2EE• DB Admin (Informix)• IT analyst/consultant• Enterprise Architect

Consumer loans

Businessloans

Mobile Bank

years of profitable growth

Active & formercustomers

Countries

9M 2018revenue

Year-on-year revenue growth

Founded Helsinki 2005

EU Banking Licence

Frankfurt Prime Standard

Year-on-year EBIT growth

• IT Hub located in Bratislava

• ~133 internal employees

• https://ferratumit.sk

• Microservices implemented in Java/Scala/TypeScript

• All deployed in OpenShift container platform

• Average latency in milliseconds to process a record within our Kafka Streams/ODS infrastructure

• Months of development to pilot production deployment

• Started August 2017

• Pilot production August 2018

• Java and Spring Boot as main backend stack

• Scala and Kafka Streams for data streams

• Avro message serialization

• Camunda BPMN for process orchestration

• Apollo GraphQL / TypeScript as API GW

Microservices• Microservice per Aggregate Root – not that small

– Customer

– Application

– Loan

• Microservices per different responsibilities

– Validations

– Basic CRUD logic

– Façade

– Wrapper

Case study – Online onboarding• Customer want’s loan for a car

• Simplified page flow would look like this:

Lead page

• Check duplicity• Basic Policy checks• Create Lead• Create Consents (marketing/GDPR…)

Verification Page

• User internal or external service for verification• Create Customer• Create User Account

Financial Page

• Use 3rd party service • Policy Checks• Scoring

Offer Page Confirmation Page

• Confirm Application• Create Loan• Create Contracts

• Store Offer• Read product configuration for the offer

Lead Page

• Check duplicity

• Basic Policy checks

• Create Lead

• Create Consents (marketing/GDPR…)

• Create Application

Verification Page

• User internal or external service for verification

• Create Customer

• Create User Account

• Update Application

Financial Page

• Use 3rd party service

• Policy Checks

• Scoring

Offer Page

• Read product configuration for the offer

• Store Offer

Confirmation Page

• Confirm Application

• Create Loan

• Create Contracts

What is long running?

Seconds Minutes Days, Weeks

“instant” flows

Automated flows

Manual flows

State handling, persistence, timeouts ,

Online Onboarding• List of MicroServices

Customer Management

Consent Management

Application Management

Loan Handling

ScoringRisk

Management

External Registry

Orchestration vs ChoreographyOrchestration

•Central brain to guide and drive the process/flow

• Invoking and combining the services

• Like conductor in the orchestra

Choreography

• Inform each microservice of it’s job

• Let it work out the details

• Like dancers in the ballet

Sequence flows for Lead Page

• Let’s start with choreography

Transformation from events to

commands

Commands instead of events

Risk ManagementWeb

Risk Management

Customer Management

loanRequested

policyChecksExecuted

leadCreated

executePolicyChecks

createLead

updateApplication

Customer Management

Risk Management

Customer Management

Who should do transformation?

• Existing domain microservice (implicit orchestrator)

– Most extensive used one

– With most logic for the given domain

– With most data for the given workflow

– In our case?

• Application Management

• New Orchestration Microservice (explicit orchestrator)

– No business logic, only flow logic

– Additional component – but improved separation of concerns

Transformation from events to

commands

Asynchronous

Orchestration

Synchronous

Orchestration

Orchestration vs Choreography

• Choreography: distributed decision making

– Every service knows what to do next…Or knows what to wait for

– It couples services together

– Bigger risk of cyclic dependencies (they depend on each other)

– Process partially works even when some microservices are down – no single point of failure

– Validations and state management are challenging

– Need for investing in monitoring and tracing features

– Compensations (order of rollback operations) are complicated

• Orchestration: centralized decision making

– Coordination component = state machine, workflow engine

– Process doesn’t work when orchestration component is down

– Risk of putting too much logic into orchestrator

– Change the flow (in one place) is much easier

– Other microservices don’t have to include state management

DeCentralized Orchestration

Onboarding Domain

Service 1 Service 2

Service N

Orchestration Service

Payments Domain

Service 1

Service 2

Service N

Collections Domain

Service 1 Service 2

Service N

Aggregate Root

Document management Domain

Service 1 Service 2

Service N

What others think about that? (1)

• Sam Newman – Building microservices (2015)– Choreography “is significantly more decoupled. If some other service needed to reach to the creation of a

customer, it just needs to subscribe to the events…”

– Orchestration “can become too much of a central governing authority. It can become the hub in the middle of a web, and a central point where logic starts to live. I have seen this approach result in a small number of smart “god” services telling anemic CRUD –based services what to do”.

– “The downside is, that the explicit view of the business process we see is now only implicitly reflected in our system”

• What does it mean for us?– Choreography is supposed to be without significant coupling

• From operational point of view – only if use asynchronous communication

• From change management point of view (logical dependencies) – you are still creating coupling

– Don’t put too much business logic into orchestration component

– Don’t make others services just dummy CRUD services

– Whole process is not explicitly visible

What others think about that? (2)

• Netflix – Conductor– With peer to peer task choreography, we found it was harder to scale with growing business needs and complexities.

Pub/sub model worked for simplest of the flows, but quickly highlighted some of the issues associated with the approach:

• Process flows are “embedded” within the code of multiple application.

• Often, there is tight coupling and assumptions around input/output, SLAs etc, making it harder to adapt to changing needs.

• Almost no way to systematically answer “how much are we done with process X”?

• What does it mean for us?

– If you have more complex flows in your business logic – use orchestration instead of choreography

– If you have complex flow logic important for your business – you have to know exactly what is going on there

Onboarding BPMN (only part)

Whole process – only onboarding domain

How to do DeCentralizedOrchestration

• Orchestration per domain

– Analyze your domain – according to the DDD

– Analyze data flows within your domain

• Orchestrator is not central brain with dummy CRUD services

– Orchestrator masters the flow logic

– Other microservices master their corresponding business logic (i.e. scoring) – thus no violation of “smart endpoints, dumb pipes” principle from M.F.

• No Central Orchestration tool for the whole architecture/company

• Lightweight orchestration tool/library

– Embeddable into your application (i.e. spring boot app)

Why workflow engine for orchestration

• The engine does all state handling and provides the required features for long running flows (monitoring, timers, versioning, …).

• You gain visibility of the flow which might be interesting during:

– requirements engineering (how to implement this?),

– development (what do I have to implement right here?),

– Testing (what it should do?) and

– operations (is everything running smooth? where exactly do we have problems?).

• You also get additional tools for advanced use cases. As an operator you could for example adjust the state for a certain flow in case of errors.

Domain Service 1

How today’s modern workflow engines works• First generation– e.g.Camunda

–“some” scalability• Optimistic locking

• Clusters

• Client-server approach

•New generation – zeebe, Netflix conductor– High scalability – scales horizontally

– Pub-sub approach

– Append only logs…

Worklowengine

Domain Service 1 Domain

Service 2

Perform command

Shared RDBMS

Store state

messaging platform

Worklow engine

Task queueOrchestrator

Task queue

Domain Service 2

Create tasks

Pool for tasks Update task status

Orchestration is not only about workflow engines

• If you need state:

– Workflow Engine (Camunda, Netflix Conductor, JBPM…)

• If you don’t need state (orchestrating multiple services in one synchronous call):

– Camel

– Spring Integration

– Lightweight ESB

Orchestration/Choreography vs Organization/Teams• If most of your projects are about implementing complex flows, orchestration can help also from

organizational perspective

– With choreography – you have to coordinate all the teams, which with lot’s of microservices is not an easy task…

– With orchestration – most changes are done in components responsible for orchestration (others are called by commands) , and only some changes in microservices which are being called

• From Project management point of view it’s easier to manage

• On the other hand – team responsible for orchestration is under bigger pressure (calling all the other components in the right way)

Workflow engine as a monitor for your choreography

Lessons Learned

What we did well

•By not putting flow logic into every microservice, we are able to build new flows quickly – changes mostly in one component – calling other microservices as needed

•Looks like a lot of compensation logic is not needed for now – we are not so big

•Our workflow/process engine contains only flow logic – all business logic is in other microservices

–But there were a lot of attempts ☺

•BPMN as the notation was a success

Challenges

•Process engine as the stateless service was not the best choice because of the frequent writes into the main entity (Application)

–Think about your domain logic constantly

•Default workflow monitoring UI is not sufficient for business people

–More complex UI needed (extend, or build a new one)

stanislav.ondac@[ferratum|gmail].com

linkedin.com/in/stanislav-ondac

@StanislavOndac

Backup

Sometimes orchestration is not needed at all• There are use-cases where just listening to the events is completely fine. Our example: Campaigns

Management

– Reading events/messages from kafka coming from different domains (client created, loan created, offer created…)

– Processing them

– Preparing reports to reward our partners

• No need for central orchestration component

Challenges of distributed computing• Client has to implement:

– Retries

– Waitings

– Timeouts

– Compensations

• Service has to implement:

– Compensation

– Idempotency

Moderná orchestrácia microslužieb - DDD community...Introduction 2. Use-Case Description 3....

Documents