Date post: | 15-Feb-2017 |
Category: |
Internet |
Upload: | nicolas-mas |
View: | 396 times |
Download: | 0 times |
How we build features at Serv.sg
in a Pandan Leaf...
January 06th 2016
Nicolas [email protected]
https://www.linkedin.com/in/nicolasmashttps://github.com/nicolasmas
Acknowledgment
“Alone we can do so little; together we can do so much”
Sandeep Isac Marco Yang Alex Hayden
Alexander Bulat Andrei Ping Yang
<ME>
Nicolas [email protected]
https://www.linkedin.com/in/nicolasmashttps://github.com/nicolasmas
</ME>
What this talk won’t do
⬗ Entertain you with funny
pictures (sorry). This will be
the only one.
⬗ Describe the tools being
used, unless there is a
specific request (let’s keep
the focus).
⬗ Guarantee that Murphy’s law
won’t impact the demo.
So what is it going to be?
⬗ What do we do at Serv.sg?
⬗ Our (current) technology stack
⬗ Problem we solve
⬗ Feature build (+ DEMO)
⬗ Back to our initial problem...
⬗ Limits
⬗ Enhancements
⬗ B Tracks: if we really have time
Disclaimer: I am a DevOps padawan, what is described tonight is probably not the best solution ever. Any suggestion is welcome~
What we do at Serv.sg
✓ Real time Mobile workforce location tracking✓ Automated scheduling of service appointments✓ Automated email / SMS reminders of upcoming appointments and contracts.✓ Manage customer data and all history of service appointments✓ Manage Services offered, including recurring contracts and packages, etc.✓ Advanced reports on employee utilization as well as business
Technologies (Stack… for now)
Key backend technology
Data storage
Key frontend technology
Technologies (DevOps)
For now… (bis)
The Problem We Hope to solve...
Feature A
Feature B
… Feature N
PRODUCT BACKLOG
QA / UAT / PRODUCTION
Build - All or Nothing
Feature A
Feature B
Monolithic App Development
Overview
JIRA-456
JIRA-456.serv.sgJIRA-456.db.serv.sg
/serv-branch-build
Branch JIRA-456
JOB JIRA-456
On “New Branch”
JOB Monitor Branches
Overview
On “Branch Deleted”
JIRA-456
JIRA-456.serv.sgJIRA-456.db.serv.sg
/serv-branch-build
Branch JIRA-456
JOB JIRA-456
JOB Monitor Branches
It all starts in Jira
⬗ The developer takes ownership of, let’s say the item “JIRA-456” during a
sprint.
⬘ Actually, he does not do it in JIRA, but from github by branching out the
codebase.
⬘ The Branch HAS TO BE named after the jira item, e.g. JIRA-456
It continues on Slack
⬗ The moment the branch is created, we can actually build the
infrastructure:⬘ In slack: “/serv-branch-build”
⬗ Slack is asking a Jenkins job to monitor the github repository for
new/deleted branches
What happens in the jenkins monitoring job
⬗ The Jenkins job will do a few things:⬘ 1. It will connect to github and get all the branches
⬘ 2. It will detect our new branch
⬘ 3. A custom jenkins job will be created for that branch
⬘ Terraform will be invocated
⬗ It will also keep you updated of the progress in Slack
The Infra is built...
...But not yet provisioned !!
Note: we use a pre-baked OS image for the instances
Building the branch
⬗ In Jenkins
Provisioning the branch
⬗ Ansible playbook⬘ Site.yml dynamically set
⬗ Roles⬘ Hostname set
⬘ Supervisord
⬘ Yum additional packages
⬘ Web conf
⬘ DB conf
⬘ DB Load
⬘ (...)
End Result
Destroying the build
⬗ The features has been validated or ditched, the branch is either merged
or destroyed on github
Note: If we need a few back and forth between the product and the dev for the feature, it’s just a matter of running the jenkins branch job to deploy again
Back in Slack
⬗ “/serv-branch-build”
⬗ AWS quick check.. the instances are terminated
Looking back at our initial problem...
⬗ Deploying by feature is not new. Immutable infra seems to be less
common. The two combined? Anyone doing it?
⬗ From a business/product perspective⬘ More responsive to bug fixing, features deliveries
⬘ No confusion on what has been deployed
⬘ The branch can be hotfixed => one step closer to CD
⬗ From a technical perspective⬘ Leveraging the cloud infrastructure, programmatically
⬘ No environment recycling or reconfiguring with the related uncertainty
⬘ As close to production as we can
⬘ A good way to craft baked OS with the latest update and test it
Limits/Challenges
⬗ Hard to keep track of the full cycle, from the OS baking to the feature
deployed:
⬘ Packer: need to manually describe the target provisioned OS. Hard to test,
need to manage the produced artifacts
⬘ Ansible: need to manually write the roles, hard to test and a pain to
maintain
⬘ Terraform: need to manually describe the infra, hard to test, painful to
maintain and evolve. Some bugs with route 53
⬘ Dependency on Jenkins
⬘ A fair amount of templating (jinja2 style) with a risk induced by the dynamic
aspect of the build (hostnames, EIP...)
⬗ Many pieces in movement, dependency
⬗ Need to be in sync with the product team to check and act upon the
feature as soon as it is available. We don’t want a stack running for ages..
Enhancement
⬗ Use a tool to orchestrate the branch + infra build aspect (Spinnaker.io?
StackStorm.io?) or build a simple micro service to do it
⬗ Choose the branch to build in slack instead of all the branches with the
pattern “JIRA-XYZ”
⬗ Improve the provisioning and better manage the ansible playbook
(Tower.io?)
⬗ How is this compatible with our next architecture steps? (micro services)
Bonus Track - Did you know ?
“The mother of all demos” http://en.wikipedia.
org/wiki/The_Mother_of_All_Demos
⬗ Douglas Engelbart's December 9, 1968
⬗ 90-minute live public demonstration of the online system, NLS, they had
been working on since 1962
⬗ 1000 attendees and a demo of almost all features we use today⬘ Mouse
⬘ hypertext, object addressing and dynamic file linking
⬘ shared-screen collaboration involving two persons at different sites
communicating over a network with audio and video interface.
http://web.stanford.
edu/dept/SUL/library/extra4/sloan/mousesite/1968Demo.html
https://www.youtube.com/playlist?list=PL76DBC8D6718B8FD3
Watch it !!
Bye Bye !