Slide 1

Agile Software Development

Robert Heim

17.01.2013

I assume: Audience has heard the term „Agile Software Development“ and has heard of practices like „Pair programming“, „Test-Driven-Development“, „Prototyping“, … I will: be abstract on the methods itself (one can easily look them up if further info needed), but be detailed on how to rate methods

Slide 2

Content Motivation

Agile Methods

◦ Attributes – What is „Agile“?

◦ Problems

◦ Evaluation criteria

◦ Examples + differences

Current movements: Programs of Agile Alliance

Discussion of Adoption

◦ Software Process Improvement

◦ Standards

◦ Example: British Telecom

◦ Empirical studies

Conclusion & Recommendations

◦ Agile Framework (process)

◦ Architecture Framework for Agile Processes (software)

Resources

Notes on some methods and additional slides for further explanations

17.01.2013

Slide 3

Motivation Problems:

Delivery on time, QM, late changes, etc.

Rapidly growing: internet and mobile apps

Need: lighter, faster, more flexible software

development process

17.01.2013 1/22

Slide 4

Content Motivation

Agile Methods

◦ Attributes – What is „Agile“?

◦ Problems

◦ Evaluation criteria

◦ Examples + differences

Current movements: Programs of Agile Alliance

Discussion of Adoption

◦ Software Process Improvement

◦ Standards

◦ Example: British Telecom

◦ Empirical studies

Conclusion & Recommendations

◦ Agile Framework (process)

◦ Architecture Framework for Agile Processes (software)

Resources

Notes on some methods and additional slides for further explanations

17.01.2013

Slide 5

Agile Software Development

„Late 1990 several methodologies began to get increasing public attention. Each had a different combination of old ideas, new ideas, and transmuted old ideas. But they all emphasized close collaboration between the programmer team and business experts; face-to-facecommunication (as more efficient than written documentation); frequent delivery of new deployable business value; tight, self-organizingteams; and ways to craft the code and the team such that the inevitable requirements churn was not a crisis.” – Agile Alliance4

17.01.2013 2/22

deployable business value = less documentation

Slide 6

Agile Methods - Attributes

Incremental (max. 3 month16)

rapid dev. cycle

cooperative

close customer

straightforward

easy to learn /

flexible methods

adaptive

last minute changes

For more see: http://agilemanifesto.org/principles.html

17.01.2013 3/22

A cycle is e.g.: 1. Functionality needed immediately, 2. Fast implementation 3. Collect feedback 4. react rapidly to business + technology changes [16] Lichter, reference models, SWC department

Slide 7

E.g. Agile Alliance◦ 2000+ papers since 2002 21

◦ Agile Conference1 2012:

~200 talks

But still: lack of scientific studiesand empirical evidence²

People not aware of approaches/suitability for real-life situations 2, 14, …

How to rate, choose and benefit from specific agile methods?

Agile Methods - Problems

17.01.2013 4/22

?

[2] Technical research center of Finland Abrahamsson… New Directions on

Agile Methods : A Comparative Analysis. Science. 2003

[14] Austrian study

Slide 8

Software dev. life-cycle

Project management support

Abstract concrete

Universally situational

Empirical evidence

Agile Methods – Evaluation Criteria²

17.01.2013 5/22

Abstract / concrete: principles <--> concrete guidance Universally = fit per se in all agile dev. situations

Slide 9

Agile Methods – Examples and

Differences²

17.01.2013 6/22

Inception = project begin/founding of project ASD = Adaptive Software Development AM = Agile Modeling DSDM = Dynamic Systems Development Method FDD = Feature Driven Development ISD = Internet Speed Development PP = Pragmatic Programming

Slide 10

Agile Methods - Choosing

Cannot answer the questions with ease:

„Should I use ASD in my upcoming project?

And which methods are suitable?“

NO STANDARD, yet(even if some ASDs are well known by many

people, e.g. XP, Scrum)

17.01.2013 7/22

Slide 11

Content Motivation

Agile Methods

◦ Attributes – What is „Agile“?

◦ Problems

◦ Evaluation criteria

◦ Examples + differences

Current movements: Programs of Agile Alliance

Discussion of Adoption

◦ Software Process Improvement

◦ Standards

◦ Example: British Telecom

◦ Empirical studies

Conclusion & Recommendations

◦ Agile Framework (process)

◦ Architecture Framework for Agile Processes (software)

Resources

Notes on some methods and additional slides for further explanations

17.01.2013

Slide 12

Agile Alliance – Current Programs1

”Supporting Agile Adoption: It’s about Change” organizational change (compare Lichter16 )

In 2012:

◦ Characteristics of agile organizations18

◦ AWG – A sustainable Engine for Enterprise Agile Adoption19

◦ Project Studies Program: longitudinal studies1

17.01.2013 8/22

Agile Alliance: a nonprofit organization with global membership, committed to advancing Agile development principles and practices Compare: my seminar 10/11 on improvement models: business has to address the culture of the organization AWG = Agile Working Group = Scrum team, with product: enterprise implementation of Agile, evidence of management support

Slide 13

Content Motivation

Agile Methods

◦ Attributes – What is „Agile“?

◦ Problems

◦ Evaluation criteria

◦ Examples + differences

Current movements: Programs of Agile Alliance

Discussion of Adoption

◦ Software Process Improvement

◦ Standards

◦ Example: British Telecom

◦ Empirical studies

Conclusion & Recommendations

◦ Agile Framework (process)

◦ Architecture Framework for Agile Processes (software)

Resources

Notes on some methods and additional slides for further explanations

17.01.2013

Slide 14

Agility and Software Process

Improvement (SPI)16

Success factors

◦ People, technology, process

◦ Team needs skills + tools required to apply process

„Never apply process models for their own sake!“16

Fitting Portfolio of Process Models (PM)

Monitor process, to see if it still fits( Quality Management e.g. software metrics12)

17.01.2013 9/22

See [7, 12]

Portfolio = Organization choose the mix of old (like CMMI) and new (like XP) models PM Selection criteria:

Organization-culture, developer-team, project-constraints, business-goals Needs to be accepted from all involved people (that it supports their daily-work)

(compare [14] Austria study)

[16] = Lichter Software Processes in an Agile World. Department did many research on Software-Process-Improvement Models, Seminar WS10/11 SCRUM in large organizations works

[ 7] = study from Pakistan, Agile Software Development : Impact on

Productivity and Quality [12] = software metrics for ASD, University of Magdeburg, Germany

Slide 15

Standards 2003: XP connectable with current ISO

“software life cycle process”11

◦ Main idea: „working software over comprehensive documentation“

◦ practical evidence missing?!

◦ Conclusion: project specific standards

Agile Alliance Program (2012): Costs of IT Projectsin coop. FASB (founded 1973):

Agile Accounting Standard◦ “define and standardize internal IT development costs […]

◦ […] for organizations that use an […] agile software development methodology”5

But: no standard for ASD it self

17.01.2013 10/22

Projects often need to fit ISO standard (due to organization, management, regulation)

11: Standards and Agile Software Development. abstract, XP and ISO norm for "software life cycle" integration, but only abstract. To fit to docu-standards-requirements: in XP as system requirement FASB (financial accounting standards board: develop generally accepted accounting principles (GAAP) within the United States in the public's interest

Slide 16

Example – British Telecom15

Environment:

◦ 2006, ~8000 IT professionals

◦ Before: waterfall with CMMI, 2 years later: DSDM + Scrum

◦ Aim/Focus: Delivering Business Value

Problems / Risks:

◦ Architecture of IT-Organization + mindsets of people have to be changed

◦ Complexity

◦ Outsourced parts of code, where no tests exists

Practice:

◦ Before: 12month + cycles

◦ Now: 90 days cycle

◦ First 3 days cross-functional teams exploring one business problem

◦ At day 90 want deployable, fully-tested solution

◦ within that 90days use internal agile methods (like 2-4week cycles).

Result: challenging, but early success and they would not turn back

17.01.2013 11/22

CMMI = Capability Maturity Model Integration DSDM = Dynamic systems development method High risk = program has business commitments to make and little scope to take extra risk of adopting new practices Deliver business value or close program if it fails to deliver value over some cycles

Slide 17

Results of (good) empirical study14

Austria 2008

Interviewed developer and manager(but answers did not differ as much as expected)

100 companies, 42% acceptance-rate

40,5% large organizations (>250)

Focus: XP, especially Pair-Programming

Results:

◦ Most know XP (~40%) and Scrum (~30%)

◦ General awareness of ASD exists

◦ But lack of knowledge of concrete practices

17.01.2013 12/22

Slide 18

Results of (good) empirical study14

Results◦ People are main obstacle to practical application of ASD,

Most important: Lack of knowledge and time

refusal of management

Concrete Reality:Agile methods used on demand

Test-first: 75% rarely use it (only when needed)

Pair-programming: 32% only for complex tasks

7,1% for tutoring (or on demand)

No really systematic approach towards agile development in practice (2008)

17.01.2013 13/22

Bad Mindset: “Adoption of some practices is enough for agile dev.” Lack of knowledge (but only 1/3 wanted more information)

Slide 19

Results of quantitative analysis7

(to be understood carefully) 2010 Pakistan

Online survey, 200 requests, 42 responses (21%), Focus: Impact on Productivity

Results:

◦ Mainly Scrum, MSF and XP known/used

◦ Fewer but competent/experienceddevelopers deliver outputs in lesser time and better quality

◦ Train people to be up to date with tools and technologies

17.01.2013 14/22

MSF = Microsoft Solutions Framework

Slide 20

Results of quantitative analysis7

(to be understood carefully) 100% do knowledge sharing as an essential

constituent of agile methodologies

Techniques used:

17.01.2013 15/22

Regression test = test to ensure that a new change has not spawn new bugs. CI: small incremental changes, developer often commit, each commit = build hole system + ideally tests TDD a little controversial to other study, cause they say test-FIRST rarely used

Slide 21

Results of quantitative analysis7

(to be understood carefully)Increase in productivity / quality

17.01.2013 16/22

Slide 22

Content Motivation

Agile Methods

◦ Attributes – What is „Agile“?

◦ Problems

◦ Evaluation criteria

◦ Examples + differences

Current movements: Programs of Agile Alliance

Discussion of Adoption

◦ Software Process Improvement

◦ Standards

◦ Example: British Telecom

◦ Empirical studies

Conclusion & Recommendations

◦ Agile Framework (process)

◦ Architecture Framework for Agile Processes (software)

Resources

Notes on some methods and additional slides for further explanations

17.01.2013

Slide 23

Conclusion

17.01.2013

Pro‘s◦ Fast delivery in praxis

◦ Easy to learn (compared to other SPI-methods)

◦ Early success/results in most companies could been shown

Con‘s◦ Missing evidence difficult method-decision

◦ Self-organizing teams not always best22

◦ Customer‘s role stressful and not sustainable over long periods10

◦ May fail in large organizations 10 (controversial) High commitment

17/22

Self-organizing not best: in complex, non-linear problem-solving situations Controversy: experienced developer tend to adopt ASD better, risk higher, see British Telcom [10 ] = 2005 What do we know about Agile Software Development?

Slide 24

Conclusion – State of Research

17.01.2013

2008 - Roadmap for empirical research:Scientific research and experimental evidence needed 22, 10

◦ Done: XP, Scrum23, especially Pair-Programming

◦ Test-driven: productivity increase less obvious and as seen rarely used in reality

Research should concentrate on: Include criteria of other fields / theories

experienced ASD teams and organizations

longitudinal studies

Prioritize management-oriented approaches

If interested: see problem definition and some practical results (mostly Scrum): 2007 - Exploratory study from Microsoft Research calling for further studies 23

18/22

Other fields: problem: SE references only SE research, but what is with not directly related to agile methods, e.g. task-technology-fit, social and organizational learning theory, knowledge-based theory of firm, … Longitudinal studies: to see effects on other components of business [22] = 2008, A Preliminary Roadmap for Empirical Research on Agile Software Development [10] = 2009, What do we know about ASD, Frauenhofer Center for Experimental SE [23] = promising Exploratory Study from Microsoft

Slide 25

Conclusion – State of Research

17.01.2013

2008 and Goal for 2015 22

19/22

Most important: Expertise = competence Research methodology Evidence / impact on industry Cohesiveness = how established ASD is [22] = 2008, A Preliminary Roadmap for Empirical Research on Agile Software Development Additional slide for explanation at end

Slide 26

Conclusion - Reality

17.01.2013

Adoption / choosing methods:

◦ ASD needs to be accepted by all participants

◦ Study characteristics of

Project

Organization structure & culture 18, 19

Regulations

compare to methods‘ characteristics to find

matches

◦ Train people (best in University20)

See [10] for further practical & realistic results

20/22

Participants: developers, managers, customers, etc. [10] = 2009, What do we know about ASD, Frauenhofer Center for Experimental SE [18] = 2012 Agile Alliance - Characteristics of Agile Organizations. 2012, [19] = 2012 Agile Alliance - AWG (Agile Working Group) A sustainable Engine for Enterprise Agile Adoption [20] = 2009 Weaving Agile Software Development Techniques into a Traditional Computer Science Curriculum

Slide 27

Content Motivation

Agile Methods

◦ Attributes – What is „Agile“?

◦ Problems

◦ Evaluation criteria

◦ Examples + differences

Current movements: Programs of Agile Alliance

Discussion of Adoption

◦ Software Process Improvement

◦ Standards

◦ Example: British Telecom

◦ Empirical studies

Conclusion & Recommendations

◦ Agile Framework (process)

◦ Architecture Framework for Agile Processes (software)

Resources

Notes on some methods and additional slides for further explanations

17.01.2013

Slide 28

Recommended Agile Framework7

(2010) Refactoring to change

internal code without

affecting

external functionality

See [7] for more details

17.01.2013 21/22

emphasis flexible design, to be able to integrate late changes

Slide 29

Problem: „ASD is an excuse for developers to implement as they like, coding away without proper planning or design […] and consequently causing suboptimal design decisions.“ 13

Approach 2010Architecture Framework for Agile processes (AFA) 13

Idea: Decoupling Components:

◦ Simple APIs of internet addressable containers

◦ Layers with different responsibilities

◦ Pro‘s: allowing changes in architecture with minimal effects on other components less testing better time-to-market

◦ Con‘s/Feature work: benchmark/performance because additional layers

Compare [6] (2010) for more detailed and practical scenarios and decision making, including freezing the architecture and how to decide when to do it, calling for more research

Recommended Software Architecture

for Agile Business Process13 (2010)

17.01.2013 22/22

Container = collection of entries accessible via basic API [13] = guys from Vienna University of Technology, Austria Coordination: e.g.: Fifo instead of Lifo can be changed, without touching algorithms for data computation

[6] = Agility and Architecture: Can they Coexist? 2010.

Slide 30

Literature / Resources [1] Agile Conference 2012 - http://agile2012.agilealliance.org/program/

[2] Abrahamsson P, Warsta J, Siponen MT, Ronkainen J. New Directions on Agile Methods : A Comparative Analysis. Science. 2003.

[3] Agile Manifesto - http://agilemanifesto.org/principles.html

[4] Agile Alliance - „What is Agile Software Development?“ -http://www.agilealliance.org/the-alliance/what-is-agile/

[5] Agile Alliance - „ Agile Accounting Standard Program“ -www.agilealliance.org/programs/agile-accounting-standard-program

[6] Abrahamsson, Pekka; Babarr, Muhammad Ali; Kruchten P. Agility and Architecture: Can they Coexist? 2010.

[7] Ahmed A, Ahmad S, Ehsan N, Mirza E, Sarwar SZ. Agile Software Development : Impact on Productivity and Quality. Development. 2010:287-291.

[8] Begel A. Usage and Perceptions of Agile Software Development in an Industrial Context : An Exploratory Study. Time. 2007:255-264.

[9] Dingsøyr T, Dybå T, Abrahamsson P. Agile 2008 Conference A Preliminary Roadmap for Empirical Research on Agile Software Development. Most. 2008:83-94.

[10] Dybå T, Dingsøyr T. What do we know about Agile Software Development?Small. 2005:0-3.

[11] Kourie DG, Watson BW. Standards and Agile Software Development. Computer. 2003:1-11.

[12] Kunz M, Dumke RR, Zenker N. Software Metrics for Agile Software Development. Work. 2008:673-678.

[13] Mordinyi R, K E, Schatten A. Towards an Architectural Framework for Agile Software Development. 2010;(section IV).

17.01.2013 L1

Slide 31

Literature / Resources [14] Schindler C. Agile Software Development Methods and Practices in Austrian

IT-Industry : Results of an Empirical Study. Technology. 2008:321-326.

[15] Agile Delivery at British Telecom 2006 -http://www.methodsandtools.com/archive/archive.php?id=43

[16] Lichter H. Software Processes in an Agile World. 2012;6(November)

[17] Agile Alliance - Resources http://www.agilealliance.org/resources

[18] Arell R, Coldewey J, Gat I, Hesselberg J. Characteristics of Agile Organizations. Methods. 2012.

[19] Arell R, Coldewey J, Gat I, Hesselberg J. A Sustainable Engine for Enterprise Agile Adoption. 2012.

[20] Maher P, Avenue EL, Louis S. Weaving Agile Software Development Techniques into a Traditional Computer Science Curriculum. 2009:0-1.

[21] Agile Alliance - Conference Papers - http://conferences.agilealliance.org/sessions

[22] 1. Dingsøyr T, Dybå T, Abrahamsson P. A Preliminary Roadmap for Empirical Research on Agile Software Development. Most. 2008:83-94.

[23] Begel A. Usage and Perceptions of Agile Software Development in an Industrial Context : An Exploratory Study. Time. 2007:255-264.

[24] IGI Global - 2005 - Agile Modeling, Agile Software Development, and Extreme Programming: The State of Research, http://www.igi-global.com/article/journal-database-management-jdm/3343

[25] AIS - Association for Information Systems - 2011 - Overview and Guidance on Agile Development in Large Organizations http://aisel.aisnet.org/cais/vol29/iss1/2/

17.01.2013 L2

Fee-based

Slide 32

APPENDIX

17.01.2013

Slide 33

(E)Xtreme Programming (XP)

Developed at Chrysler ~1996 – 2000

Collection of concrete best practices

E.g.

◦ Short iterations

◦ Pair programming

◦ Rapid feedback / close customer

◦ Communication

◦ Continuous refactoring + integration + testing

◦ …

17.01.2013 A1

Slide 34

Scrum

~1995

Manage Software development process

Empirical base, focus on

◦ Flexibility

◦ Adaptability

◦ Productivity

◦ Frequent management activities + Sprints

◦ Developer chooses specific

techniques/practices for implementation

17.01.2013 A2

Slide 35

Adaptive Software Development

(ASD) ~2000

Adaptive/incremental paradigm instead of

waterfall

Constant prototyping

Framework for guidance to prevent chaos,

but not suppressing emergence and

creativity

17.01.2013 A3

Slide 36

Agile Modeling (AM)

~2002

Focus: modeling and cultural principles

Modeling using agile philosophy

Advanced models

Aim: amount of models and

documentation as low as possible

17.01.2013 A4

Slide 37

Crystal Family

~1998

Different methods

Select best fitting based on „color-rating“

(size and criticality)

Tailoring methods to fit needs

Integrate other agile methods like

XP/Scrum

17.01.2013 A5

Slide 38

Dynamic Systems Development

Method (DSDM)

One of the first truly agile methods

(~1994)

Normally: fixed functionality + adjust

time/resources

Here: fix time/resources and adjust

functionality

17.01.2013 A6

Slide 39

Feature Driven Development (FDD)

~2002

Process oriented for business systems

Design and building phases

Iterative

Quality aspects in process monitoring

of progress

17.01.2013 A7

Slide 40

Internet Speed Development (ISD)

~2001

Need: Fast releases in a chaotic fast

moving process

Descriptive, management-oriented

framework to handle fast iterations:

◦ Time-drivers

◦ quality dependencies

◦ good people = less process

17.01.2013 A8

Slide 41

Pragmatic Programming (PP)

~2000

Collection of best practices

~70 so called „tips“ focusing day-to-day

problems

Incremental, iterative development,

testing, user-centered design

17.01.2013 A9

Slide 42

Description of current status and

suggested goal for 2015 22

17.01.2013 A10