why agile?

Tathagat Varma

Sr. Director

Yahoo!

the world around us…yesterday!

Microsoft Windows timeline

https://en.wikipedia.org/wiki/Timeline_of_Microsoft_Windows

Other major OS and tools timeline

https://en.wikipedia.org/wiki/Solaris_(operating_system) https://en.wikipedia.org/wiki/Bugzilla

A typical support timeline

http://itconvergence.blogspot.in/2012/10/oow-12-elison-extradata-oracle-r122.html

As a contrast, what are consumer internet companies doing?

Continuous Integration -> Continuous Delivery -> Continuous Deployment

On ‘good days’, Flickr releases a new version every half an hour (Jun 20, 2005)

IMVU pushes a revision of code to the website every nine minutes (Feb 10, 2009)

The other day we passed product release number 25,000 for WordPress. That means we’ve averaged about 16 product releases a day, every day for the last four and a half years! (May 19, 2010)

A new version of Google Chrome now due every six weeks (Jul 22, 2010)

Facebook does code push twice a day (Aug 4, 2012)

adoption @ net speed!!!

https://lh4.googleusercontent.com/-SQ1ugOgu8Ds/TimaHjH0VpI/AAAAAAAAApU/za0BhEjhoio/w402/google%2Bplus%2Bgrowth%2B20mil.png http://thesamerowdycrowd.wordpress.com/2011/12/26/a-moment-of-our-time/

darwin at work on internet

http://www.flickr.com/photos/stabilo-boss/93136022/sizes/o/in/photostream/

2006 2009

http://rossdawsonblog.com/weblog/archives/2009/05/rapid_turnover.html

yet, our product development sucks!

http://blog.amplifiedanalytics.com/2011/07/musing-on-difference-between-successful-product-innovation/ http://www.nickblack.org/2009/10/how-brand-trust-affects-new-products.html

Let’s understand the ‘craft’ first…

Sheer joy of making things

Pleasure of things that are useful to other people

Fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles

Joys of always learning, which sprints from the non-repeating nature of the task

Delight of working in such a tractable medium

The Mythical Man Month – Fred Brooks, 1975

software development life cycles

Ad-hoc

Serial

Iterative

Incremental

Iterative/Incremental

risk management in SDLCs

notion of ‘progress’ vs. time

Waterfall Model

Wrongly inspired by assembly-line manufacturing processes of the day

Economics supported “measure twice, cut once” leading to up-front planning and BDUF

Single-pass, sequential process with hand-offs and feedback loops between adjoining phases

Transition to next phase only upon completion of current phase

Waterfall Software Development

Picture from http://damonpoole.blogspot.in/2009/07/traditional-development-game-of.html

Limitations and Assumptions

1. Wrong analogy: Software development ≠ Production 2. Customers know EVERYTHING upfront and that requirement won’t change3. Legacy from the past: implicitly assumes CPU time is costly, so focuses on

doing everything upfront to minimize ‘machine time’ for trial and error4. “Wicked Problem”: Designers and developers know how exactly how to

build5. Very long feedback cycles not suitable for today’s pace of innovation

As a result, software is…

Late

Buggy

Costly

and the costs…?

http://leadinganswers.typepad.com/leading_answers/estimating/http://www.agileforall.com/dyk/

Holy Grail of Software Development

Better: higher quality, more reliability, higher performance, more usable…

Faster: speedier development

Cheaper: no budget overruns

But the reality?

Preamble to Agile Movement

Software Crisis, 1965-85: The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem. — Edsger Dijkstra, The Humble Programmer

Software Crisis

The causes of the software crisis were linked to the overall complexity of hardware and the software development process. The crisis manifested itself in several ways:

Projects running over-budget. Projects running over-time. Software was very inefficient. Software was of low quality. Software often did not meet requirements. Projects were unmanageable and code difficult to

maintain. Software was never delivered.

and the response?

Frameworks, Standards and Certifications

…and the result?…good start…

…but poor finish!

and sadly, none of these came out of ‘process

factories’…

1975-2000•2000: Baidu•1997: Yandex•1994: Yahoo!, Amazon, NetScape,

•1986: Pixar•1984: Sybase•1983: Intuit, Borland, •1982: Sun, Symantec, Adobe, EA

•1980: Informix•1979: EMC•1977: Oracle•1976: CA, Apple•1975: Microsoft

2000-2010• 2010: Pinterest, SnapDeal, • 2009: Square, Quora, Sina Weibo• 2008: Groupon, AirBnB, GoGo• 2007: Dropbox, Zynga, Flipkart,

InMobi, Hulu, Tumblr, • 2006: Twitter, SlideShare, Badoo

Spotify• 2005: YouTube, Renren• 2004: Facebook• 2003: Myspace, Skype, Rovio,

Gameforge, • 2002: LinkedIn• 2001: StumbleUpon, Mail.ru

2011 -• Instagram

Why?

Process: Long-lead development process ineffective in a dynamic and global world

Management: Command and control model unsuitable for fostering collaboration required to solve complex problems

Technology: Advancements in computers, compiler technology and debugging and testing tools greatly improved the economics of software development

Innovation: in the age of hyper-innovation, old processes were simply ineffective

What is the most important part in these two machines?

“The Brakes!!!”

They let you go faster…

Agility vs. Discipline?

http://www.ibm.com/developerworks/rational/library/edge/08/feb08/lines_barnes_holmes_ambler/

Advent of Agile and Lean Methodologies

1970: Royce critiques Waterfall and offers improvement ideas 1986: Barry Boehm proposes Spiral Model 1971: Harlan Mills proposes Incremental Development 1987: Cleanroom Software engineering 1991: Sashimi Overlapping Waterfall Model 1992: Crystal family of methodologies 1994: DSDM 1995: Scrum 1996: Rational Unified Process framework 1997: Feature Driven Development 1999: Extreme Programming Explained 2001: Agile Manifesto is born 2003: Lean Software Development 2005: PM Declaration of Interdependence 2007: Kanban-based software engineering 2008: Lean Startup 2009: Scrumban 20xx: Something new !?! (hopefully!)

What is agile really all about?

Agile Businesse

s

Self-organizin

g x-functional

Teams

Motivated Individuals

• Higher ROI• Faster time to market• Better User Experience

• Shorter feedback cycle • Manage changing

priorities• Increased productivity

• Empowered individuals• Collaboration • Democratic decision-

making and transparency

Why is it so hard?

52%Organizational

Culture

39%

Resistance to Change

34%

Management Support

feedback loop in agile lifecycles

from daily builds to project

Scrum

What’s happening here?

http://ayagebeely.blogspot.in/2008_08_01_archive.html

Feedback Loops in Traditional Techniques vs.

Agile Techniques

Agile Development Value Proposition

http://www.versionone.com/Agile101/Agile_Benefits.asp

Does Agile work?

http://www.bigvisible.com/2009/12/taking-agile-beyond-faster/http://www.testingthefuture.net/page/2/

does iterating help?

http://viniciusvacanti.com/2011/12/12/when-do-you-throw-in-the-towel-on-your-struggling-project/

are small teams more productive?

http://drewcrawfordapps.com/2.0/the-agility-of-small-teams/

does colocation impact team performance?

http://sloanreview.mit.edu/the-magazine/2009-summer/50412/how-to-manage-virtual-teams/

is small batch size faster?

http://www.andrejkoelewijn.com/wp/2011/06/30/is-team-productivity-a-responsibility-of-the-product-owner/

Let’s build a car

…and I need it delivered…next week!

www.wikispeed.com

The Wikispeed Process

At WIKISPEED, some of our projects move more than 10,000% faster than industry norms because of our blend of Agile, Lean, Scrum, and Extreme Programming/Manufacturing practices.

Team WIKISPEED uses methods developed by the fastest-moving software companies. In fact, in many ways we have more in common with Google or Twitter than with GM or Toyota.

Manufacturing and old-thought software teams gather requirements, design the solution, build the solution, test the solution, then deliver the solution. In existing automotive companies, the design portion of that process alone takes 3 to 12 years, and then the vehicle design is built for 5 to 14 years. This means it is possible to buy a brand new car from a dealer and that car represents the engineering team's understanding of what the customer might have wanted 26 years ago!

Team WIKISPEED follows the model of Agile software teams, compressing the entire development cycle into one-week "sprints." We iterate the entire car every 7 days, meaning that every 7 days we reevaluate each part of the car and reinvent the highest-priority aspects, instead of waiting 8 to 26 years to upgrade. 

Wikispeed uses…

Lean Software Design: Use less stuff

XP: Pairing and Swarming

Agile: Reducing costs to make changes

Scrum: Clearly defined team roles and responsibilities

TDD: start with failing tests and develop solutions

OOP: contract-first development

Recap

agile ≠ Faster, but Sooner

agile ≠ No planning, but Adaptive Planning

agile ≠ More work, but ‘Done’

agile ≠ No documentation, but Just Enough

agile doesn’t just change the development process, but bring a radical change in organizational culture, leadership and management practices that is more in line with business needs and social values and norms of today

It’s not about the method!

A photographer went to a socialite party in New York. As he entered the front door, the host said ‘I love your pictures – they’re wonderful; you must have a fantastic camera.’

He said nothing until dinner was finished, then: ‘That was a wonderful dinner; you must have a terrific stove.’

– Sam Haskins

http://www.haskins.com/ImageShop/Image_Shop_60s/60s_Books_A.Image_01.html

Connect

Blog: http://managewell.net

Twitter: http://twitter.com/TathagatVarma

Presentations: http://slideshare.net/managewell