Date post: | 10-May-2015 |
Category: |
Software |
Upload: | andreas-grabner |
View: | 573 times |
Download: | 1 times |
Performance Quality Metrics along CD
Pipeline@grabnerandi
http://apmblog.compuware.com
And just very recently @ Euro Song Contest
What you should check out …
Tonight we are here for …
Commit Stage• Compile• Execute Unit Test• Code Analysis• Build installers
Automated Acceptance
Testing
Automated Capacity Testing
Manual testing• Key showcases• Exploratory testing Release
Cont. Delivery / Deploy
Or – The Evolution of Software Engineering
Who is doing it? How many successful deployments can they do?
300 Deployments / Year
50-60 Deployments / Day
10+ Deployments / Day
Every 11.6 seconds
More on Amazons Story
75% fewer outages since 2006
90% fewer outage minutes
~0.001% of deployments cause a problem
Instantaneous automatic rollback
Deploying every 11.6s
Testing is Important – and gives Confidence
But are we ready for “The Real” world?
Measure Performance during the game
Ball Possession: 40 : 60Fouls: 0 : 0Score: 0 : 0
Minute 1 - 5
Measure Performance during the game
Minute 6 - 35
Ball Possession: 80 : 20Fouls: 2 : 12Score: 0 : 0
Deep Dive Analysis
Options “To Fix” the situation
Not always a happy ending
Minute 90
Ball Possession: 80 : 20Fouls: 4 : 25Score: 3 : 0
FRUSTRATED FANS!!
23
How does that relate to
Software?
How does that relate to
Software?
From Deploy to …
Deploy Promotion/Event Problems Ops Playbook War Room
Timeline
The “War Room” – back then
'Houston, we have a problem‘NASA Mission Control Center, Apollo 13, 1970
The “War Room” – back then
'Houston, we have a problem‘NASA Mission Control Center, Apollo 13, 1970
The “War Room” – NOW
Facebook – December 2012
The “War Room” – NOW
Facebook – December 2012
4 Situations on
WHY this happens,
HOW to avoid it
4 Situations on
WHY this happens,
HOW to avoid it
“Blindly” (Re)use Existing
Components
Requirement: We need a report
Using Hibernate results in 4k+ SQL Statements to display 3 items!
Hibernate Executes 4k+ Statements
Individual Execution VERY
FAST
But Total SUM takes 6s
Requirement: We need a fancy UI
Using Telerik Controls Results in 9s for Data-Binding of UI Controls
#1: Slow Stored ProcedureDepending on Request
execution time of this SP varies between 1 and 7.5s
#2: 240! Similar SQL StatementsMost of these 240! Statements are
not prepared and just differ in things like Column Names
Metrics: # Total SQLs# SQLs / Web Request# Same SQLs / Request
Transferred Rows
Test: With realistic Data
Dev: “Learn” Frameworks
Implementation Flaws
Business Impact requires Action!
Solution: Cache to the RESCUE!!
Implementation and Rollout
Implemented InMemory Cache
Worked well in Load Testing
Result: Out of Memory Crashes!!
Still crashes
Problem fixed!Fixed Version Deployed
Metrics: Heap Size, # Objects Allocated,# Objects in Cache
Cache Hit Ratio
Test: With realistic Data
12 000 000 $
#No “Agile” Deployment
Ad on air
Availability dropped to 0%
Load Spike resulted in Unavailability
Alternative: “GoDaddy goes DevOps”
Response time improved 4x
1h before SuperBowl KickOff
1h after Game ended
Behind the Scenes
Metrics: AvailabilityPage Size, # Objects
# Hosts, # Connections
DevOps: “Feature” Switches
#Push without a Plan
Mobile Landing Page of Super Bowl Ad
434 Resources in total on that page:230 JPEGs, 75 PNGs, 50 GIFs, …
Total size of ~ 20MB
m.store.com redirects to www.store.com
ALL CSS and JS files are redirected to the www domain
This is a lot of time “wasted” especially on high latency mobile
connections
Metrics: Load Time, # Resources (Images, …),
# HTTP 3xx, 4xx, 5xx
Dev: Build for Mobile
Test: Test on Mobile
Ops: Monitor Mobile
#1: Define Measures# of Requests / User
# of Log Messages
# of Exceptions
# Objects Allocated
# Objects In Cache
Cache Hit Ratio
# of Images
# of SQLs
# SQLs per RequestAvailability
# HTTP 3xx, 4xx
Page Size
#2: Automate
Commit Stage• Compile• Execute Unit Test• Code Analysis• Build installers
Automated Acceptance
Testing
Automated Capacity Testing
Manual testing• Key showcases• Exploratory testing Release
Unit & Integration Tests
Functional Tests
Performance TestsProductionMonitoring
Functional Tests
57
#3: Share Results
#4: Integrate
If we do all that
CANCELLED
Which gives you more time for the real important things in life …
FREE Products & More Info
• dynaTrace Enterprise– Full End-to-End Visibility in your Java, .NET, PHP Apps
– Sign up for a 15 Days Free Trial on http://compuwareapm.com
• dynaTrace AJAX Edition– Browser Diagnostics for IE + FF
– Download @ http://ajax.dynatrace.com
• Our Blog: http://apmblog.compuware.com