6/28/17, 11)02 AMCloud Native Go
Page 1 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Cloud Native Go
Building Scalable, Resilient Microservices for the Cloudin Go
1 / 29
6/28/17, 11)02 AMCloud Native Go
Page 2 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Agenda1. Introduction2. Boring Stuff3. Live Demos4. Q & A
2 / 29
Cloud Native Go
Building Scalable, Resilient Microservices for the Cloudin Go
1 / 29
6/28/17, 11)02 AMCloud Native Go
Page 3 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Introduction
Wrote a few books (2 fantasy, 15+ .NET/C#, Go)Maker, Tinkerer, LinguistTaught people how migrate/build cloud native for PivotalLead Software Engineer for Capital OneTwitter: @KevinHoffman, github autodidaddict
Go to gopherize.megopherize.me for your Gopher Avatar
3 / 29
Agenda1. Introduction2. Boring Stuff3. Live Demos4. Q & A
2 / 29
6/28/17, 11)02 AMCloud Native Go
Page 4 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Why Go?FastLow memory, CPU, and Disk footprintDocker images from SCRATCHFit more containers per node/VMSingle, no-dependency binaryBeautiful, simple languageShort learning curve
4 / 29
Introduction
Wrote a few books (2 fantasy, 15+ .NET/C#, Go)Maker, Tinkerer, LinguistTaught people how migrate/build cloud native for PivotalLead Software Engineer for Capital OneTwitter: @KevinHoffman, github autodidaddict
Go to gopherize.megopherize.me for your Gopher Avatar
3 / 29
6/28/17, 11)02 AMCloud Native Go
Page 5 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
What is Cloud Native?API FirstDependency ManagementDesign, Build, Release, RunConfig, Credentials, and CodeLogsDisposabilityBacking ServicesEnvironment ParityAdministrative ProcessesPort BindingStateless ProcessesConcurrencyTelemetryAuthentication and Authorization
5 / 29
Why Go?FastLow memory, CPU, and Disk footprintDocker images from SCRATCHFit more containers per node/VMSingle, no-dependency binaryBeautiful, simple languageShort learning curve
4 / 29
6/28/17, 11)02 AMCloud Native Go
Page 6 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Microservices Shopping ListHTTP ServerRouting
URL path variablesQuery strings
JSON Encode/DecodeMiddleware (logging, security, etc)
6 / 29
What is Cloud Native?API FirstDependency ManagementDesign, Build, Release, RunConfig, Credentials, and CodeLogsDisposabilityBacking ServicesEnvironment ParityAdministrative ProcessesPort BindingStateless ProcessesConcurrencyTelemetryAuthentication and Authorization
5 / 29
6/28/17, 11)02 AMCloud Native Go
Page 7 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
HTTP Server package main
import (...)
func main() { port := os.Getenv("PORT") if len(port) == 0 { port = "8080" }
mux := http.NewServeMux() mux.HandleFunc("/", hello)
n := negroni.Classic() n.UseHandler(mux) hostString := fmt.Sprintf(":%s", port) n.Run(hostString)}
func hello(res http.ResponseWriter, req *http.Request) { fmt.Fprintln(res, "Hello from Go!")}
7 / 29
Microservices Shopping ListHTTP ServerRouting
URL path variablesQuery strings
JSON Encode/DecodeMiddleware (logging, security, etc)
6 / 29
6/28/17, 11)02 AMCloud Native Go
Page 8 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Routing Routing with Gorilla Mux
r := mux.NewRouter()r.HandleFunc("/products/{key}", ProductHandler)r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler)r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler)
s := r.PathPrefix("/zombies").Subrouter()s.HandleFunc("/", ZombiesHandler)s.HandleFunc("/sightings", NewSightingHandler).Methods("POST")s.HandleFunc("/{key}", QueryZombieHandler).Methods("GET")
URLs:
/zombies/zombies/sightings (POST)/zombies/bob (GET)
8 / 29
HTTP Server package main
import (...)
func main() { port := os.Getenv("PORT") if len(port) == 0 { port = "8080" }
mux := http.NewServeMux() mux.HandleFunc("/", hello)
n := negroni.Classic() n.UseHandler(mux) hostString := fmt.Sprintf(":%s", port) n.Run(hostString)}
func hello(res http.ResponseWriter, req *http.Request) { fmt.Fprintln(res, "Hello from Go!")}
7 / 29
6/28/17, 11)02 AMCloud Native Go
Page 9 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Routing Accessing Route Data
vars := mux.Vars(req)zombieID := vars["zombie-key"]
Building URLs
r := mux.NewRouter()r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article")...url, err := r.Get("article").URL("category", "technology", "id", "42")
Builds - /articles/technology/42
9 / 29
Routing Routing with Gorilla Mux
r := mux.NewRouter()r.HandleFunc("/products/{key}", ProductHandler)r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler)r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler)
s := r.PathPrefix("/zombies").Subrouter()s.HandleFunc("/", ZombiesHandler)s.HandleFunc("/sightings", NewSightingHandler).Methods("POST")s.HandleFunc("/{key}", QueryZombieHandler).Methods("GET")
URLs:
/zombies/zombies/sightings (POST)/zombies/bob (GET)
8 / 29
6/28/17, 11)02 AMCloud Native Go
Page 10 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
JSON Marshaling Read JSON from Body:
payload, _ := ioutil.ReadAll(req.Body)var newMatchRequest newMatchRequesterr := json.Unmarshal(payload, &newMatchRequest)
Write JSON to Response:
var mr newMatchResponsemr.copyMatch(newMatch)w.Header().Add("Location", "/matches/"+newMatch.ID)formatter.JSON(w, http.StatusCreated, &mr)
10 / 29
Routing Accessing Route Data
vars := mux.Vars(req)zombieID := vars["zombie-key"]
Building URLs
r := mux.NewRouter()r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article")...url, err := r.Get("article").URL("category", "technology", "id", "42")
Builds - /articles/technology/42
9 / 29
6/28/17, 11)02 AMCloud Native Go
Page 11 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Middleware apiRouter := mux.NewRouter()apiRouter.HandleFunc("/api/post", apiPostHandler(formatter)).Methods("POST")
router.PathPrefix("/api").Handler(negroni.New( negroni.HandlerFunc(isAuthorized(formatter)), negroni.Wrap(apiRouter),))
func isAuthorized(formatter *render.Render) negroni.HandlerFunc { return func(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { providedKey := r.Header.Get(APIKey) if providedKey == "" { formatter.JSON(w, http.StatusUnauthorized, struct{ Error string }{"Unauthorized." } else if providedKey != apikey { formatter.JSON(w, http.StatusForbidden, struct{ Error string }{"Insufficient credentials." } else { next(w, r) } }}
11 / 29
JSON Marshaling Read JSON from Body:
payload, _ := ioutil.ReadAll(req.Body)var newMatchRequest newMatchRequesterr := json.Unmarshal(payload, &newMatchRequest)
Write JSON to Response:
var mr newMatchResponsemr.copyMatch(newMatch)w.Header().Add("Location", "/matches/"+newMatch.ID)formatter.JSON(w, http.StatusCreated, &mr)
10 / 29
6/28/17, 11)02 AMCloud Native Go
Page 12 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Shopping List ✅ HTTP Server✅ Routing
✅ URL path variables✅ Query strings
✅ JSON Encode/Decode✅ Middleware (logging, security, etc)
12 / 29
Middleware apiRouter := mux.NewRouter()apiRouter.HandleFunc("/api/post", apiPostHandler(formatter)).Methods("POST")
router.PathPrefix("/api").Handler(negroni.New( negroni.HandlerFunc(isAuthorized(formatter)), negroni.Wrap(apiRouter),))
func isAuthorized(formatter *render.Render) negroni.HandlerFunc { return func(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { providedKey := r.Header.Get(APIKey) if providedKey == "" { formatter.JSON(w, http.StatusUnauthorized, struct{ Error string }{"Unauthorized." } else if providedKey != apikey { formatter.JSON(w, http.StatusForbidden, struct{ Error string }{"Insufficient credentials." } else { next(w, r) } }}
11 / 29
6/28/17, 11)02 AMCloud Native Go
Page 13 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
We're Doing This All Wrong
13 / 29
Shopping List ✅ HTTP Server✅ Routing
✅ URL path variables✅ Query strings
✅ JSON Encode/Decode✅ Middleware (logging, security, etc)
12 / 29
6/28/17, 11)02 AMCloud Native Go
Page 14 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Do One Thing Great
A fox knows many things, but a hedgehog one important thing -Archilochus
14 / 29
We're Doing This All Wrong
13 / 29
6/28/17, 11)02 AMCloud Native Go
Page 15 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Hexagonal ArchitecturePorts and Adapters
15 / 29
Do One Thing Great
A fox knows many things, but a hedgehog one important thing -Archilochus
14 / 29
6/28/17, 11)02 AMCloud Native Go
Page 16 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports and Adapters in GoBuild your Core Functionality FirstDecouple HTTP, JSON, Headers, Security from Core FunctionShould be able to test your core function in isolationCode is blissfully ignorant of source and nature of inputs and ultimatedestination of replies
16 / 29
Hexagonal ArchitecturePorts and Adapters
15 / 29
6/28/17, 11)02 AMCloud Native Go
Page 17 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService Ecosystem
17 / 29
Ports and Adapters in GoBuild your Core Functionality FirstDecouple HTTP, JSON, Headers, Security from Core FunctionShould be able to test your core function in isolationCode is blissfully ignorant of source and nature of inputs and ultimatedestination of replies
16 / 29
6/28/17, 11)02 AMCloud Native Go
Page 18 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI Gateways
17 / 29
Ports & Adapters in a μService Ecosystem
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 19 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator Services
17 / 29
Ports & Adapters in a μService EcosystemAPI Gateways
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 20 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
17 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator Services
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 21 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / Translators
17 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 22 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service Discovery
17 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / Translators
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 23 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service DiscoveryExternal Configuration
17 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service Discovery
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 24 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service DiscoveryExternal ConfigurationAdapter paths through system
Gateways, ProxiesTranslators on Data Streams / Queues
17 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service DiscoveryExternal Configuration
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 25 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service DiscoveryExternal ConfigurationAdapter paths through system
Gateways, ProxiesTranslators on Data Streams / Queues
Hexagonal arch inside and outside of services
17 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service DiscoveryExternal ConfigurationAdapter paths through system
Gateways, ProxiesTranslators on Data Streams / Queues
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 26 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
The Shopping List We IgnoreSecurityMonitoringLog AggregationZero Downtime DeploymentBuild/Deploy AutomationTracingMetricsAuditAutomatic / Dynamic Horizontal ScalingConfigurationSecrets ManagementDiscovery
18 / 29
Ports & Adapters in a μService EcosystemAPI GatewaysAggregator ServicesMessage Brokers
TopicsPub/Sub
ACL / TranslatorsDynamic Service DiscoveryExternal ConfigurationAdapter paths through system
Gateways, ProxiesTranslators on Data Streams / Queues
Hexagonal arch inside and outside of services
17 / 29
6/28/17, 11)02 AMCloud Native Go
Page 27 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Starting at the CoreDefine what we're buildingDefine why we're building itWho are we building it for?Build the coreAdd onion layers around the core
Ports and AdaptersRemember we're building an ecosystem, not hello world
19 / 29
The Shopping List We IgnoreSecurityMonitoringLog AggregationZero Downtime DeploymentBuild/Deploy AutomationTracingMetricsAuditAutomatic / Dynamic Horizontal ScalingConfigurationSecrets ManagementDiscovery
18 / 29
6/28/17, 11)02 AMCloud Native Go
Page 28 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
The Go Shopping Sample
20 / 29
Starting at the CoreDefine what we're buildingDefine why we're building itWho are we building it for?Build the coreAdd onion layers around the core
Ports and AdaptersRemember we're building an ecosystem, not hello world
19 / 29
6/28/17, 11)02 AMCloud Native Go
Page 29 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Go Shopping Sample - Techgo micro Framework
https://micro.mu/Ports and Adapters are services
Not boilerplate inside serviceEcosystem-aware from bottom-upDynamic Service DiscoverygRPC TransportRESTful Aggregate API... and much more
21 / 29
The Go Shopping Sample
20 / 29
6/28/17, 11)02 AMCloud Native Go
Page 30 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Framework vs LibraryOpinionated vs boilerplateSpend boilerplate to insulate from opinions
Mitigate ceremony with code generation?Go-kit - suite of libraries, incur BP costGo-micro - opinionated, less BP
Spring Boot - example of opinionated framework
22 / 29
Go Shopping Sample - Techgo micro Framework
https://micro.mu/Ports and Adapters are services
Not boilerplate inside serviceEcosystem-aware from bottom-upDynamic Service DiscoverygRPC TransportRESTful Aggregate API... and much more
21 / 29
6/28/17, 11)02 AMCloud Native Go
Page 31 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API FirstAPI First is NOT:
RESTful Endpoint FirstSwagger-FirstOne API to Rule them All
API First is:
A strict, semantically versioned, interaction specification.A team and organization-wide disciplineDifferent consumers, channels may need different APIs
Ports and Adapters
23 / 29
Framework vs LibraryOpinionated vs boilerplateSpend boilerplate to insulate from opinions
Mitigate ceremony with code generation?Go-kit - suite of libraries, incur BP costGo-micro - opinionated, less BP
Spring Boot - example of opinionated framework
22 / 29
6/28/17, 11)02 AMCloud Native Go
Page 32 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Defining the Warehouse Service APIProtocol Buffer IDLgRPC Service
syntax = "proto3";
package warehouse;
service Warehouse { rpc GetWarehouseDetails(DetailsRequest) returns (DetailsResponse);}
message DetailsRequest { string sku = 1;}
message DetailsResponse { WarehouseDetails details = 1;}
message WarehouseDetails { string sku = 1; uint32 stock_remaining = 2; string manufacturer = 3; string model_number = 4;}
24 / 29
API FirstAPI First is NOT:
RESTful Endpoint FirstSwagger-FirstOne API to Rule them All
API First is:
A strict, semantically versioned, interaction specification.A team and organization-wide disciplineDifferent consumers, channels may need different APIs
Ports and Adapters
23 / 29
6/28/17, 11)02 AMCloud Native Go
Page 33 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Service Implementation
package service
import ( "github.com/autodidaddict/go-shopping/warehouse/proto" "golang.org/x/net/context")
type warehouseService struct{}
// NewWarehouseService returns an instance of a warehouse handlerfunc NewWarehouseService() warehouse.WarehouseHandler { return &warehouseService{}}
func (w *warehouseService) GetWarehouseDetails(ctx context.Context, request *warehouse.DetailsRequest, response *warehouse.DetailsResponse) error {
response.Manufacturer = "TOSHIBA" response.ModelNumber = "T-1000" response.SKU = request.SKU response.StockRemaining = 35 return nil}
25 / 29
Defining the Warehouse Service APIProtocol Buffer IDLgRPC Service
syntax = "proto3";
package warehouse;
service Warehouse { rpc GetWarehouseDetails(DetailsRequest) returns (DetailsResponse);}
message DetailsRequest { string sku = 1;}
message DetailsResponse { WarehouseDetails details = 1;}
message WarehouseDetails { string sku = 1; uint32 stock_remaining = 2; string manufacturer = 3; string model_number = 4;}
24 / 29
6/28/17, 11)02 AMCloud Native Go
Page 34 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Let's do some Live DemosWhat could possibly go wrong?
26 / 29
Service Implementation
package service
import ( "github.com/autodidaddict/go-shopping/warehouse/proto" "golang.org/x/net/context")
type warehouseService struct{}
// NewWarehouseService returns an instance of a warehouse handlerfunc NewWarehouseService() warehouse.WarehouseHandler { return &warehouseService{}}
func (w *warehouseService) GetWarehouseDetails(ctx context.Context, request *warehouse.DetailsRequest, response *warehouse.DetailsResponse) error {
response.Manufacturer = "TOSHIBA" response.ModelNumber = "T-1000" response.SKU = request.SKU response.StockRemaining = 35 return nil}
25 / 29
6/28/17, 11)02 AMCloud Native Go
Page 35 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Lessons LearnedQuestion EverythingBeware of Protobuf Default ValuesService-first developmentCode is the easiest and simplest part of Microservices
NFRs are everything - logging, metrics, alerts, discovery, health,autoscale, async messaging, etc.
Decide where you want to be on the opinion vs boilerplate curveGo is an ideal microservices development language
27 / 29
Let's do some Live DemosWhat could possibly go wrong?
26 / 29
6/28/17, 11)02 AMCloud Native Go
Page 36 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Next Steps Create ServicesTest everythingAutomate everythingDeploy all the timeTry out go-microTry out go-kitGo Shopping http://github.com/autodidaddict/go-shoppingThese slides https://github.com/autodidaddict/cng-presentation
28 / 29
Lessons LearnedQuestion EverythingBeware of Protobuf Default ValuesService-first developmentCode is the easiest and simplest part of Microservices
NFRs are everything - logging, metrics, alerts, discovery, health,autoscale, async messaging, etc.
Decide where you want to be on the opinion vs boilerplate curveGo is an ideal microservices development language
27 / 29
6/28/17, 11)02 AMCloud Native Go
Page 37 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Q & A
29 / 29
Next Steps Create ServicesTest everythingAutomate everythingDeploy all the timeTry out go-microTry out go-kitGo Shopping http://github.com/autodidaddict/go-shoppingThese slides https://github.com/autodidaddict/cng-presentation
28 / 29
6/28/17, 11)02 AMCloud Native Go
Page 38 of 38file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Q & A
29 / 29