How Database DevOps Will Help You
To Stay Ahead Of The Competition
Bob Walker, Solutions Architect at Octopus Deploy
Agenda
• Problem to solve
• DevOps
• Benefits
• Roadblocks
• Questions
What is DevOps trying
to Solve?
Previous Work Experience
• Lifting Controls for Oil Terminals
• Loan Origination System
• Electronic Pollbook
Software Development Life Cycle
Development Team
Operations Team
Knowledge Silos
• Development Team• Creates roadmap• Feature requirements• Knows how to fix the application
• Operations Team• Performance statistics• Support tickets submitted• Incidents
Typical Communication Between Teams
• “This service is down please restart it”
• “Your service is consuming 16 GB of RAM, fix it now”
• “I have a call from a customer and they say your
application is slow”
• “I got paged last night because this SQL Job
failed. Fix it.”
Developer’s Mindset
A Lack of Trust
Lack Of Trust
Slows Everything
Down
Real-World Example
• Loan Origination System
• Used Redgate Schema compare to compare test
and prod
• Scripts dropped into shared folder to be ran
• 2ish hour deployment
Real-World Example
• Lifting Controls Application
• Manually Written Database Scripts
• Manual Deployments
• 2 AM Saturday Morning Deployments
• 2 – 4 hour deployments on average
Expected
Perfection
Big Bang Deployments
• Multiple Sprints prior to going to production
• Multiple KanBan cards bundled together prior to
going to production
• Long Waterfall release cycles (quarter or months)
Perfect Storm
• Lifting Control Application: Quarterly Release with
six month rotation
• Loan Origination System: Quarterly Release
• Electronic Pollbook: Three Month - Eight Month
Release
Everyone Is Unhappy
• Business Owners• “This isn’t going to work now, we need to add the following features to
make this viable”• “Fine. Fine. Fine. Deploy it as is. I’ll add it my next release doc”
• Users• “This isn’t what I wanted”• How long until the application is no longer relevant• Head to competitors
• Security• Data Compliance• Patches
DevOps to the rescue!
Right?!?!? Right?!?!?!
DevOps: Development
and Operations Teams
Working Together
Towards a Common
Goal
Common Goal Examples
• 1 deployment a month
• 1 deployment a week
• Zero Downtime Deployments
• 10 deployments a day
• Fixing all critical bugs within 2 business days
• Performance tests before each release
• Decrease deployment time
• Increase response time on critical methods
• Decrease resource usage
Benefits
Automated Database Deployments
• Loan Origination System
• Database Developer, Lead Developer, DBA, Architect and Redgate Consultants collaborate on a initial process
• Pilot Team adopted process and tooling
• Iterate through the process
• 2ish hour deployments went down to 10ish minute deployments
Less Time Fixing Bugs
• 3200 schema objects
• 2 to 3 emergency fixes post release
• Missed views, stored proc updates, tables, etc
• Post automation this went down to zero
Multi-tenant Database Deployments
• Electronic Pollbook Reporting Application
• 100+ jurisdictions with unique databases
• Unique delta script per jurisdiction
• Prep-work minimum two days
• Deployment minimum four hours
• Post Automation 20 – 30 minutes of prep, 30 minute deployment
Fixing Critical Bug
• Customer reported bug to their process through
support channels at Octopus Deploy Tuesday
morning
• Discussed bug during the day
• Fix released late Tuesday night
• In customer’s hands by Wednesday morning
Better Performance
• Loan Origination System
• Endpoint averaging 2000 requests per minute
• After deployment average response time went
from 20 ms to 200 ms
• Identify slow performance and deploy release
before first support ticket came in
Staying Ahead Of Competition
• Respond to customer feedback
• Fix critical bugs
• Experiment and A/B testing
• Keep company relevant
Higher Employee Utilization
Getting Started
Octopus Deploy and Redgate to the
rescue!
Hold your horses…
Start Small
• Attend meetings
• Discuss issues and complaints
• Listen and Learn
• Pick a common problem
• Use existing tooling
Go Bigger
• Solve a larger problem
• Pilot Application / Group
• New Tooling
• New Processes
Tooling
• Build: TeamCity / VSTS / Jenkins / Bamboo
• Unit Testing: NUnit / MSTest / XUnit
• Integration Testing: Selenium / Postman
• Deploy: Octopus Deploy / Redgate (for DBs) / VSTS Release / DACPAC
• Monitoring: NewRelic / DataDog / DynaTrace / SQL Sentry
• Error Logging: Splunk / Seq / PaperTrail
DevOps is
Both People
and Tools
Roadblocks
People
• “We’ve been doing it this way for 10 years”
• “Sure what we have is a little bumpy, but it works”
• “I’m too busy to change”
• “This was tried once before and it failed”
• “What you proposed won’t work for me or my
team, we’re special”
And some people…
Time
• Time to implement changes
• Each new change requires multiple iterations
• Deadlines and other conflicting priorities
Cost
• Tools cost money
• Cost per development hour
• Consultants
• Months or years before an ROI
Breaking Through Roadblocks
• Technology is the easy part
• Pilot team / project
• Small changes
• Don’t shoot for perfection at first
• Provide clear and measurable stats
• Be Transparent
Anyquestions?
Thank You!