11

COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

LIFECYCLE BY DESIGN

Enabling Collaboration across Development, Test and Production Teams

Wolfgang Gottesheim

22

Application Lifecycle Management…?Application Lifecycle Management…?

33

Application Lifecycle ManagementApplication Lifecycle Management

Application lifecycle management (ALM) is the product lifecycle management (governance, development, and maintenance) of application software. It encompasses requirements management, software architecture, computer programming, software testing, software maintenance, change management, project management, and release management.

Source: http://en.wikipedia.org/wiki/Application_lifecycle_managementaccessed Dec 16, 2013

44

The Application LifecycleThe Application Lifecycle

Source: http://www.teamquest.com/solutions/itil/application-management/

55

Project deliverablesProject deliverables

…and what you get…and what you get

What you set out to build…What you set out to build…

66

What we tried to do…What we tried to do…

• Update our Customer Self-Service Portal

77

Our (Business) RequirementsOur (Business) Requirements

• “Provide a modern user experience”

– aka “Get the Community from Web 1.0 to Web 2.0”

– New features: AJAXified editors, Popups when someone else is editing the same page, notifications about new comments, auto refresh in the background,…

88

What went wrong?

Our mistakes: We left PERFORMANCE outWe thought about new features instead of thinking about the user experience

99

1010

DesignDesign

• We wanted to continue using Confluence

– 3rd party software major design decisions made elsewhere

– Extended by custom plugins ensure compatibility

• Our mistake: assuming that resource usage will remain stable in the new version

1111

Build/Integrate/TestBuild/Integrate/Test

• Testing was focused on FUNCTIONALITY

– “Does the site work as expected?”

– Tests in small-scale staging environment some findings disregarded because of environmental differences

• Our mistake: we didn’t test from the end-user perspective in a realistic environment

1212

DeployDeploy

• Actual rollout was documented and well-tested in staging environment

• Also part of deployment phase: User Training

– We thought the new features are so obvious that people will find them

• Our mistake: we didn’t make sure that features are accessible

1313

OperateOperate

• Mistakes we made earlier added up…

– Slow response times

– High resource usage (CPU, memory)

– Timeouts

– Crashes

– User complaints

• Blame it on IT?

1414

What problems did we see?

1515

Resource Pool ExhaustionResource Pool Exhaustion

1616

Compatibility Issues with custom pluginsCompatibility Issues with custom plugins

1717

Requesting too much data from DBRequesting too much data from DB

1818

Oversized PagesOversized Pages

17! JS Files – 1.7MB in Size

Useless Information!Even might be a security risk!

1919

CachesCaches

62! Resources not cached

49! Resources with short expiration

2020

3rd Party Content3rd Party Content

2121

2222

OptimizeOptimize

• Monitor the impact of new features on server-side performance and end-user experience

• Monitor actual usage and consider tradeoff between performance and features

• Our success: after rollout, we were able to identify the bottlenecks and make necessary changes

2323

2424

Very “expensive” to work on these issuesVery “expensive” to work on these issues

~80% of problems

caused by ~20% patterns

YES we know this

BUT

2525

Lots of Problems that could have been avoidedLots of Problems that could have been avoided

• BUT WHY are we still dragging them along across the lifecycle – up into production?

?

2626

Requirements EXCLUDED in Stories / TasksRequirements EXCLUDED in Stories / Tasks

PerformancePerformance ScalabilityScalability

TestabilityTestability

DeployabilityDeployability

DeployabilityDeployability

2727

Test and Ops EXCLUDED from Agile ProcessTest and Ops EXCLUDED from Agile Process

Stand-UpsStand-Ups Sharing ToolsSharing Tools

FeedbackFeedback

2828

Automation EXCLUDESAutomation EXCLUDES

PerformancePerformance ScalabilityScalability

Shared ToolsShared Tools Automatic FeedbackAutomatic Feedback

2929

Measurement: Define KPIs accepted by all teamsMeasurement: Define KPIs accepted by all teams

# of SQL Executions

# of Log Lines

MBs / Uses

Time for Deployment

Time for Rollback

Response TimesPerf Test Code Coverage

3030

DevOps Automation in ActionDevOps Automation in Action

• Most problems can be identified in CI

12 0 120ms

3 1 68ms

Build 20 testPurchase OK

testSearch OK

Build 17 testPurchase OK

testSearch OK

Build 18 testPurchase FAILED

testSearch OK

Build 19 testPurchase OK

testSearch OK

Build # Test Case Status # SQL # Excep CPU

12 0 120ms

3 1 68ms

12 5 60ms

3 1 68ms

75 0 230ms

3 1 68ms

Test Framework Results Architectural Data

We identified a regresesion

Problem solved

Lets look behind the scenes

Exceptions probably reason for failed tests

Problem fixed but now we have an architectural regression

Problem fixed but now we have an architectural regression

Now we have the functional and architectural confidence

3131

Performance Focus in Test AutomationPerformance Focus in Test Automation

Analyzing All Unit / Performance Tests

Analyzing Metrics such as DB Exec

Count

Jump in DB Callsfrom one Build to the next

3232

Performance Focus in Test AutomationPerformance Focus in Test Automation

Cross Impact of KPIs

3333

Performance Focus in Test AutomationPerformance Focus in Test Automation

Embed your Architectural Results in Jenkins

3434

Performance Focus in Test AutomationPerformance Focus in Test Automation

Here is the difference!

Compare Build that shows BAD Behavior! With Build that shows GOOD Behavior!

3535

Performance Focus in Test AutomationPerformance Focus in Test Automation

CalculateUserStats is the new Plugin that causes problems

36© 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved

36

Participate in Compuware APM Discussion Forumsapmcommunity.compuware.com

Like us on Facebookfacebook.com/CompuwareAPM

Join our LinkedIn groupCompuware APM User Group

Follow us on Twittertwitter.com/CompuwareAPM

Read our Bloghttp://apmblog.compuware.com

Watch our Videos & product Demosyoutube.com/Compuware

www.compuware.com/APM

Thank YouThank You

37© 2011 Compuware Corporation — All Rights Reserved © 2011 Compuware Corporation — All Rights Reserved

37 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE