1 2 FA C T O R A P PS O F T W A R E E N G I N E E R I N G PA T T E R N S

A B O U T M E

• Docker, 12 Factor Apps, Massive Cloud Scaling is a bit new to me.

• Discovered this over the last 3 months through researching new workflows with Docker.

• Know a thing or two but am no Linux guru.

1 2 FA C T O R A P P ?

C O N C E P T S

C L O U D S C A L A B I L I T Y

C L O U D S C A L A B I L I T Y = H O R I Z O N TA L S C A L A B I L I T Y

… A A S M O D E L S… A S A S E R V I C E M O D E L S

AS A SERVICE MODELS

AS A SERVICE MODELS

– 1 2 FA C T O R A P P M A N I F E S T O

http://12factor.net

W H AT A R E T H E 1 2 FA C T O R S ?

• They are guidelines for writing scalable SaaS apps

W H AT I S A 1 2 FA C T O R A P P ?

• Manifesto written around 2012

• by Adam Wiggins

• Heroku ( Early cloud PaaS )

http://12factor.net

W H Y S H O U L D I C A R E ?

W H Y S H O U L D I C A R E ?

• Many start up apps get written quickly

• Limited scalability

• Tangled dependencies

• Need a rewrite to scale easily

O H A N D …

If you use docker in production you probably already have a 12 factor app!

I W I S H I K N E W T H I S B E F O R E I S TA R T E D W I T H D O C K E R !

L E S S O N S L E A R N T

T H E FA C T O R S

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

1 . C O D E B A S EO N E C O D E B A S E M A N Y D E P L O Y S

C O D E B A S E

• codebase = repo

• one repo - many deploys

• app != many repos

• many repos = distributed system

2 . D E P E N D E N C I E SE X P L I C I T LY D E C L A R E A N D I S O L A T E

D E P E N D E N C I E S

Explicitly declare and isolate dependencies

1. Declare dependencies in a manifest

2. Use isolation tools

3. Specific versions are important

4. Avoid shelling to unbundled system tools.

D E P E N D E N C I E S

Examples:

Dependency Manifest = Gemfile

Isolation tools = bundle exec

gem "redis-rails", "~> 3.2.3"

3 . C O N F I GS T O R E I N T H E E N V I R O N M E N T

C O N F I G

• Config is the specific information required to host a deployment of your codebase

• Does not include things like routes etc.

• Mainly database credentials, paths, resource urls etc.

C O N F I G

Store config in the environment.

• Keep your config outside the app

• No config in git

• Open source test

C O N F I G

Use environment vars

Can you make your repo open source today?

ENV[‘DATABASE_URL’]

4 . B A C K I N G S E R V I C E SA S A T TA C H E D R E S O U R C E S

4 . B A C K I N G S E R V I C E S

Treat backing services as attached resources

What’s a backing service?

• Datastore • SMTP • Caching systems • Amazon S3

Make no distinction between local and third party services

5 . B U I L D R E L E A S E R U NS T R I C T LY S E PA R A T E B U I L D & R U N S TA G E S

B U I L D , R E L E A S E , R U N

BUILD = codebase + dependencies + assets

RELEASE = BUILD + config

RUN = run process against RELEASE

B U I L D , R E L E A S E , R U N

B U I L D R E L E A S E R U N

• Strict separation between stages

• Cannot change code at runtime

• Rollback = just use the last release instead.

• Release has unique release ID

6 . P R O C E S S E SE X E C U T E T H E A P P A S O N E O R M O R E S TA T E L E S S P R O C E S S E S

P R O C E S S E S

• Does the running of the release

• Is stateless

• Shares nothing with other processes

• Uses single transaction only caches

• Session db storage …over sticky sessions

• Asset pre-compilation …over runtime calculation

7 . P O R T B I N D I N GE X P O R T S E R V I C E S V I A P O R T B I N D I N G

P O R T B I N D I N G

http://192.168.123.45 : 5555

The contract with the executive environment is binding to a port to serve requests.

P O R T B I N D I N G

E-Commerce System:

http:// 192.168.123.45 : 5555

Booking System:

http:// 23.123.65.48 : 3306

8 . C O N C U R R E N C YS C A L E O U T V I A T H E P R O C E S S M O D E L

C O N C U R R E N C Y

• Scale out via the process model

• Processes are first class citizens.

• Assign work to a process type (web, db, worker etc.)

• Can then handle own multiplexing

• Processes don’t daemonize or write PID files.

• Instead, use system process manager

C O N C U R R E N C Y

Scale out by running multiple processes of different types

9 . D I S P O S A B I L I T YM A X I M I S E R O B U S T N E S S W I T H FA S T S TA R T U P A N D G R A C E F U L S H U T D O W N

D I S P O S A B I L I T Y

Processes are disposable

• Start quickly

• Shut down gracefully

• Be robust against sudden death

D I S P O S A B I L I T Y

Example: Worker Process

• Return the current job to the job queue

• All jobs are reentrant

• Or at least idempotent

1 0 . D E V P R O D PA R I T YK E E P D E V E L O P M E N T, S TA G I N G , A N D P R O D U C T I O N A S S I M I L A R A S P O S S I B L E

D E V P R O D PA R I T Y

Gaps exist between development and production:

• Time (days to push)

• Personnel (dev vs ops)

• Tools

D E V P R O D PA R I T Y

Need to shorten the gaps!

• CI & deploy ASAP after writing code

• Get Developers involved in Operations

• Environments should be as similar as possible:

• Eg. Resist the urge to use different backing services between development and production

1 1 . L O G ST R E A T L O G S A S E V E N T S T R E A M S

L O G S

Within your app treat logs as event streams

• Don’t route or store logs in files

• Stream to stout instead and be captured and handled by the environment

1 2 . A D M I N P R O C E S S E SR U N A D M I N / M A N A G E M E N T TA S K S A S O N E - O F F P R O C E S S E S

A D M I N P R O C E S S E S

One-off admin tasks include:

• Database migrations

• Console

• One-time scripts

A D M I N P R O C E S S E S

• Run as separate process

• Run against the same release

• Admin code ships with app code

T H E FA C T O R S

1. Codebase

2. Dependencies

3. Config

4. Backing Services

5. Build, Release Run

6. Processes

7. Port Binding

8. Concurrency

9. Disposability

10. Dev/Prod Parity

11. Logs

12. Admin Processes

W H AT M I G H T T H I S A L L L O O K L I K E ?

T H A N K Y O UQ U E S T I O N S ?