Date post: | 15-Jan-2015 |
Category: |
Software |
Upload: | yoav-avrahami |
View: | 365 times |
Download: | 0 times |
So#ware Architecture
Reversim Summit 2014
Why do we need architecture?
What it is good for?
Is there a role for an Architect?
Architecture is both the process and the product of planning, designing, and construcBng
buildings and other physical structures.
Architecture is both the process and the product of planning, designing, and construcBng so#ware products and other so#ware systems.
Is this soDware architecture?
• It looks like a blueprint
Is this soDware architecture?
However • The main funcBon of a building is sta4c • The main funcBon of soDware is dynamic
– Good soDware react to user input and different external events – SoDware evolves with Bme
A Blueprint is not a good representa4on of so#ware architecture!
Is this soDware architecture?
What is SoDware Architecture?
• Code dependencies • TesBng dependencies • Deployment dependencies
• VPNs • VLANs • Routers • Switches
• CP vs AP • Data layout • OpBmizaBon
• Process interacBons • SLA • Data centers • Failure recovery
RunBme Data
Build Network
RunBme Architecture
It’s about the runBme construcBon of soDware
• A soDware system is built of OS processes – Deployed on one or more hosts
• The Processes collabora4on is the so#ware system – InteracBons via RPC, web services, MQs, BUS, shared memory, files, etc.
• Process interacBons with data storage – Databases, NoSQL engines, disks, data store APIs
• Reverse Proxies, Service Bus – Non-‐funcBonal facilitators of interacBons between processes
ConsideraBons for RunBme Architecture
• Resource Management and Capacity Planning – CPU, disk & network (IO), resource pools (threads, DB connecBons)
• Service Level Requirements / Agreements – Availability requirements, DRP, performance, etc.
• OrganizaBonal Structure • When shit hits the fans
– Self healing systems vs terminal failures – Containment of failures
• “Browser land” – Security & cross domain constraints
• CAP Theorem • Note: cache is not part of an architecture – it is an opBmizaBon
Data Architecture
It’s about your data • What is your data? • What is the structure of the data?
– RelaBonal, columnar, doc, key/value, events, search index, queues, graph, logs, unstructured, pictures, CRDTs …
Create Cart
Add item
Add item
Remove item
checkout
key value key value
key value key value
key value
Column 1 2 Noddle 1 Pig
Spider Cookie
A 3 Killer B 4 Dipface C 5 Swamp D 6 Donkey E 7 Wooly F 8 Weasel
H G
10 9
J I
Column 2
Column 3
Data Architecture
How do you use your data? • Read paoern?
– RelaBonal, object, CQRS (Query SegregaBon), Bme series, event sourcing, map reduce, full table scans
– OperaBonal, offline and realBme – the three scopes of data analysis
• Write paoern? – OperaBonal, ETLs, data imports
• Search paoern? – By field filters, full text search, links (between items), relaBons
• Data Mutability – Is your data mutable (e.g. update & delete operaBons)?
ConsideraBons for Data Architecture
• CP vs AP – Consistency vs Availability – CAP theorem mandates you cannot get both (unless you allow split brain…) – CRDTs -‐ CommutaBve (or Conflict free) Replicated Data Types
• Eventual Consistency – Can you model your system to cope with a certain level of inconsistency?
• Performance vs Size vs Features – OpBmize for fast reads / fast writes / huge size / complex queries & MR
• DRP – What happens if your engine dies? – What happens if your data get poisoned?
Build Architecture
It’s about how you Build your code Build ConsideraBons • Dependencies
– Code, Jar, Build, TesBng & Deployment dependencies
• Build pipeline – Building dependencies in order – Building against the right dependencies
• Minimize building and tesBng Bme • Version management
– Major.minor.build.patch.fix.test…. – Do we need versions at all?
Build Architecture
Dependencies • The root of all evil – reuse
– Dependencies are introduced because component A uses component B – Without reuse, there are no dependencies
• Diamond problem – Manual dependency progression (Maven, Ivy) – AutomaBc dependency progression (conBnuous delivery) – “Pick your poison” dependency progression (NPM)
• Dependencies complexity – Complexity is an exponenBal funcBon
of the number of components
D
C B
A 1.1
A 1.0
Build Architecture
It’s about how you Deploy your code • Deployment
– Decoupling deployment from feature release – Automated deployment – Zero-‐downBme deployment
• Rolling updates, parallel clusters (Netlix) – MulBple-‐environment support
• Development, Debugging, TesBng, Staging, ProducBon
• Post-‐Deployment – A/B tests – Feature Toggles – Immune system
• Startup tests, post deployment monitoring
– Automated rollback
What about Code Level Architecture?
I do not believe there is “code-‐level architecture” • Coding is Engineering
– WriBng modular code, maintainable code, testable code – Design paoerns are a theoreBcal tool
• Architecture is about real-‐worlds constraints – In an ideal world – infinite space, infinite performance, infinite consistency
magic database – do we need architecture?
• Coding by itself is in a mathemaBcal exercise, in a theoreBcal domain. – Only when introducing “real-‐world” constraints do we need architecture
The Architect Role
Architect Architecture
!=
Anyone can do architecture
Everyone should do architecture
So what is the role of the Architect?
Lets consider a product
A Hotel Management system
Frontend UI
Hotel & Room Management ReservaBons Checkout
Backoffice UI
Frontend Server
MySQL ElasBc Search
Payment Provider
The teams Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
ReservaBons Checkout
MySQL Payment Provider
A-Team
Team B
Team C
And a Feature
Search for available rooms • Query the Hotel & Room
Management service for the rooms that fit the search – By hotel star raBng, review
score, etc.
• Query the ReservaBon service to find available rooms – By availability of rooms, based
on current reservaBons
The Hotel Search Feature? Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
ReservaBons Checkout
MySQL Payment Provider
A-Team
Team B
Team C
Are you
real???
1. Frontend calls Hotels & Rooms Man. for the search, gets back a few pages of rooms.
2. Then, it calls ReservaBons with the results to filter out unavailable rooms
The Hotel Search Feature? Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
ReservaBons Checkout
MySQL Payment Provider
A-Team
Team B
Team C
No Hacks Here!
1. Frontend calls Hotels & Rooms Man. for the search
2. Hotels & Rooms Man. searches for the right rooms
3. Then, it calls ReservaBons with the results to filter out unavailable rooms
What is a Hack?
What is a Hack?
• Wikipedia – Hack is a soluBon to a problem, doing a task, or fixing a system that is
inefficient, inelegant, or even unfathomable, but which nevertheless (more or less) works
• What team C considers a hack
– The idea that the search service calls the inventory service with the found rooms to check availability, merges the response and filters the results
The Hotel Search Feature? Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
ReservaBons Checkout
MySQL Payment Provider
A-Team
Team B
Team C
1. Frontend calls ReservaBons for the search
2. ReservaBons finds all available rooms on the specified dates
3. Then, it calls Hotels & Room Man. to filter and sort by the available rooms
The Architect Role
• Did we define the right services? – Should the Hotel & Rooms Management and the ReservaBons Service be
separate services?
• Did we define the right teams? – Can the problem between the two services be caused because those are
owned by two different teams?
• IdenBfy the “hacks” – Teams tend to resist doing what they perceive as hacks – Each team tries to force the hacks out of their domain,
turning hacks into integra4on problems – Which hacks are real hacks (we don’t want those) – Which hack are redefini4ons of parts (we want those)
The Architect Role
Assuming we have the right teams and services • Help the teams select a soluBon
– Using the architecture consideraBons we talked about
• Redefine the services – A Service is defined by the use-‐cases it supports
• And not by some theoreBcal idea of the team
– If a Service needs to support search by doing another hackish request to another service – it is not a hack, but rather a redefini4on of the service
• Beware, this statement does not mean a team should accept any hack!
The Architect Role
The Architect role is to connect the Micro vs Macro Macro – whole product / company / soluBon • Business Requirements
– Are we building the right product? What do want to get out of it? – Evolve with the business
• Product DefiniBon – Assess features starBng with user stories and screen mockups
• Real-‐world constraints – All the consideraBons we talked about
The Architect role is to help teams sort the right path between requirement and constraints
The Architect Role
The Architect role is to connect the Micro vs Macro Micro – one part / component / team
• A Component should do one well defined thing – Handles one specific role – Does the team understand the role?
• Coherence – Is the part / component well defined?
• The role is defined by the requirements of the whole
The Architect role is to help the team understand their part in the whole
The Architect Role
Connect the Micro and the Macro • All for One and One for All
– Teams and Components should support the business
– Align the teams as part of the whole
• Challenge the teams – Their architecture – Their part in the whole architecture
The Architecture Process
Business Requirements Product DefiniBon
Compile
End 2 End Feature Flows
DecomposiBon to teams &
components
Redefine Component Boundaries
Architect facilitates the process Teams do the architecture
• RunBme • Data • Build
Real-‐World Constraints
How teams & components collaborate to build the end-‐2-‐end product
Ques4ons? anyone?
My SoluBon
Broaden the scope and allow some inconsistency • Reflect the room availability in the search service
– We store availability twice (in reservaBons and search) – may get our of sync – Have the reservaBon service update the search on room orders
• Handle the inconsistency – On opening a room page, or on ordering a page, we need to recheck the room
availability
What do we gain? • Simpler search flow
– Search is used more compared to ordering – should be simpler
• More robust system – Using async updates decouples failure and dependencies
ReservaBons Checkout
MySQL Payment Provider
My SoluBon -‐ Order Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
A-Team
Team B
Team C
Order
Are you insane?
1. On ReservaBon, ReservaBons noBfy Hotel & Room Man. • Async, low priority
2. Hotel & Room Man. stores inventory per room
– We lose consistency!!!
Me? Inventory?
My SoluBon -‐ Search Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
ReservaBons Checkout
MySQL Payment Provider
A-Team
Team B
Team C
Search
1. Search is done with the Hotel & Room Management system, including room availability
– Result may be inconsistent
My SoluBon – Showing Room Page Frontend
UI Backoffice
UI
Frontend Server
Hotel & Room Management
ElasBc Search
ReservaBons Checkout
MySQL Payment Provider
A-Team
Team B
Team C
1. Because we have a potenBally inconsistent state, as a user views a room page, we have to recheck availability
Recheck availability
It’s not our job!