CS 5500 – FOUNDATIONS OF SOFTWARE

ENGINEERING

INTRODUCTION

M. Weintraub

& F. Tip

Thanks go to Andreas Zeller for allowing incorporation of his

materials

https://course.ccs.neu.edu/cs5500

SOFTWARE ENGINEERING

The application of systematic, disciplined, quantifiable approaches to the development, operation, and maintenance of software.

IEEE Standard Glossary of Software Engineering Terminology, IEEE Standard 610.12-1990, 1990.

SOFTWARE CAUTIONARY TALES

3

SOFTWARE DISASTERS: DENVER INTERNATIONAL AIRPORT

4

Ambitious new automated system for baggage handling

SOFTWARE DISASTERS: DENVER AIRPORT

5

Ambitious new automated system for baggage handling

One of the more notorious examples of project failure

Resulted in the newly complete airport sitting idle for 16 months while engineers worked on getting the system to work

Added approximately $560M to cost of the airport

Feature article in Scientific American titled the “Software’s Chronic Crisis”

http://calleam.com/WTPF/?page_id=2086http://users.csc.calpoly.edu/~jdalbey/SWE/Papers/SciAmGibbs/SciAmGibbs.html

AN ADVANCED IDEA:DENVER INTERNATIONAL AIRPORT

6

DENVER INTERNATIONAL AIRPORT

7

▪ Approved for construction in 1989

▪ First major airport to be built in the United States in over 20 years.

▪ Biggest US Airport

▪ Built on 54.05 square miles of land (Twice the size of Manhattan!)

▪ At the time 6th busiest US Airport and 10th busiest world-wise

▪ Three terminals + five runways

BAE (BOEING AIRCRAFT EQUIPMENT) CONTRACT

8

Original assumption:

▪ Every company builds its own baggage transport system

▪ United (70% Denver traffic) was the only to begin planning and contracted with BAE

The Idea: First Fully Automated Baggage System

▪ United insisted on fully automated (like it had at SFO)

▪ Replace legacy tug-driven baggage handling

▪ Expected benefits: reduced air pollution, fewer traffic issues within airport tunnel system, and lower labor expense

Later, Denver airport extended contract to entire airport – three times original size

SCOPE

9

▪ 20 miles of track

▪ 6 miles of conveyor belts

▪ 56 laser arrays that read bar coded tags

▪ 400 frequency readers

▪ 3,100 standard size baggage ‘Telecars’

▪ 450 6.5 ft by 4 ft oversize cars

▪ 55 separate computers

THE SYSTEM

10

THE SYSTEM

Unloading a Telecar to a Belt Loading a Telecar

11

RISK #1: THE TIMEFRAME

12

BAE started work 17 months before scheduled opening October 31, 2003

RISK #1: THE TIMEFRAME

13

BAE started work 17 months before scheduled opening October 31, 2003

In Munich, engineers working a similar system spent two years just testing the system

Ran the system test 24x7 for six months before the airport opened

OTHER RISKS

14

Accommodating Legacy

Most of buildings holding the system were already done (without consideration for this new system), so BAE had to accommodate them as they were (sharp turns, narrow corridors…)

Learning Curve

BAE paid little attention to the German sister project and devised system from scratch

Symptom of NIH Syndrome (Not Invented Here)

Since there’s be no major airport projects in almost a generation, there’s no experience base to draw upon.

Complexity

There are many (moving) pieces to this system.

The Unknown

This was a first of its kind system

THE FINAL BLUNDER (GREAT PR IDEA???)

15

The decision to broadcast the preliminary test of the “revolutionary” new baggage system on national television

DISASTER

▪ Carts jammed together

▪ Damaged luggage everywhere, some bags literally split in half

▪ Tattered remains of clothing strewn about caused subsequent carts to derail

▪ Carts got stuck in narrow corridors

▪ Wind blew light baggage from carts

Cart Issues

▪ 5% of the labels were read correctly

▪ Half the luggage that survived the ordeal ended up at the wrong terminal

▪ Normal network load was 95%

System Issues

16

CONSEQUENCES

Airport opening delayed four times – overall, sixteen months late

New engineering firm

split system in three (one per terminal)

implemented manual backup system

BAE went bankrupt

Overall damage: $1.3B

System scuttled 2005

17

NBC News, 2010

WHAT WENT WRONG?

18

EverythingYou can read more at: http://calleam.com/WTPF/wp-content/uploads/articles/DIABaggage.pdf

A SAMPLING OF ISSUES FROM 2018

19

EXAMPLES CONTINUED

EXAMPLES CONTINUED

TRICENTIS 2018 SOFTWARE FAILURE WATCH (REPORT)

https://www.tricentis.com/wp-content/uploads/2019/01/Software-Fails-Watch-5th-edition.pdf

Estimated losses from software failures (USD)

1,715,430,778,504

Amount of world’s population software failures

affected (at least)

50%

Even if these magnitudes are off by half, the impacts are huge.

LOOKING AT DEVELOPMENT PROJECTS –STANDISH GROUP’S CHAOS STUDY (2015)

Success defined as On Time, On Budget, and On Target.

… means the project was resolved within a reasonable estimated time, stayed within budget, and contained a good number of the estimated features and functions

Source: https://www.standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf

RESULTS WHEN INCLUDING USER SATISFACTION IN THE METRIC

Success defined as On Time, On Budget, with a satisfactory result.

…. means the project was resolved within a reasonable estimated time, stayed within budget, and delivered customer and user satisfaction regardless of the original scope.

Source: https://www.standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf

SOME FRIGHTENING PREDICTIONS

25

In the coming years organizations are expected to spend more than $1 trillion annually on IT hardware, software, and services worldwide. (IDC)

Based on the Standish Chaos numbers, of these projects, between 5 and 15% will likely be considered complete failures.

Many others will arrive very late and require substantial rework to be useful.

That’s a lot of waste.

#1 REASON WHY PROJECTS FAIL (GLASS’ LAW)

26

Requirement deficienciesare the prime source

of project failures.

OTHER REASONS PROJECTS FAIL

1. Unrealistic project goals

2. Unmanaged risks

3. Poor project planning and estimating

4. Inaccurate estimates of needed resources

5. Poor communication among project constituents

6. Lack of user participation

Project Management Issues

1. Use too many new technologies

2. Wrong or inefficient set of development tools

3. Poor testing methodology

4. Inadequate test coverage

Technical Team Issues

27

5. Incompetent team members

6. Poor software process

Business

#1 SUCCESS FACTOR: INVOLVING THE RIGHT GROUPS AND GOOD COMMUNICATIONS CHANNELS

28

Competitors

Finance

Lawyers

Executives

Users

Tech TeamsClient/Owners

Bosses

Product/Marketing

Other Sponsors

Operations

EACH CONSTITUENT MAY HAVE A DIFFERENT PERSPECTIVE

29

SUCCESS FACTORS

1. Executive Support

2. User Participation

3. Experienced Project Management

4. Clear Business Objectives and Success Criteria

5. Adequate Prioritization of Requirements

Business Oriented

1. Standardized Infrastructure

2. Formalized Methodology

3. Reliable and Realistic Estimates

4. Competent Workers

5. Small, well-defined projects with incremental deliverables

Technology Team Oriented

30

FIRST QUESTION ON THE TEST

31

Writing good code is a necessary condition,

but it’s not enough to guarantee success

WHAT MIGHT SUCCESS MEAN?

32

1. Functionality: The system must do what it needs to do.

2. Usability: The users of the system must be able to use it effectively and efficiently.

3. Maintainability: Changes or repairs must be able to be made directly and implemented without untoward impacts

4. Efficiency: The system must make effective use of resources

5. Dependability/ Reliability/Resiliency: The system should be available and remain working for a given period of time. When faced with failures or bad data, performance should not be impacted, or at least degrade gracefully.

An ideal solution maximizes these five dimensions.

DEFINING A SUCCESSFUL SOLUTION – THE MANAGEMENT VIEW

1. Actually work

2. Data needed for billing can be accurately and quickly collected

3. Adapts to changes in market conditions

4. Satisfies any regulators

5. Users see the system as a net plus rather than a net negative

6. Whoever runs the system can do so

7. Is (easily) extensible or repairable

8. Is defined, constructed, and deployed in a reasonable timeframe within an acceptable budget

9. Runs cost-effectively

10. Puts us ahead, even, or closer to the competition.

33

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

Obviously delivering a quality product to customers is important.

But, quality has different meanings to different people.

Quality measures must be specified up-front to

insure the team is pursuing the right targets.

34

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

2. High Quality Software is Possible

35

It is hard to produce high-quality software.

Modern software engineering techniques are shown to meet reasonable quality goals.

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

2. High Quality Sotware is Possible

3. Give Products to Customers Early

36

Many projects fail because customers don’t get to see anything until it’s too late to do much about it.

Given it’s nearly impossible to specify a project completely up-front, involving customers early on provides critical feedback.

Early engagement often tests whether

the perceived needs are the actual needs.

This principle applies to other project constituents.

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

2. High Quality Sotware is Possible

3. Give Products to Customers Early

4. Use an Appropriate Software Process

37

There are many ways to run a project.

The key factors are the availability and understanding level of the requirements and the number of releases.

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

2. High Quality Sotware is Possible

3. Give Products to Customers Early

4. Use an Appropriate Software Process

5. Minimize Intellectual Distance

38

There should be a natural mapping from the real-world problem and the software solution.

The closer the two are, the easier it is to develop and maintain the solution.

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

2. High Quality Sotware is Possible

3. Give Products to Customers Early

4. Use an Appropriate Software Process

5. Minimize Intellectual Distance

6. Inspect Code

39

Everything – tech specs, test plans,

documents, and code – should be

reviewed early and often.

Early inspection is essential to finding

problems as early as possible, which

improves solution quality and decreases project cost.

SOFTWARE ENGINEERING PRINCIPLES

1. Make Quality Job #1

2. High Quality Sotware is Possible

3. Give Products to Customers Early

4. Use an Appropriate Software Process

5. Minimize Intellectual Distance

6. Inspect Code

7. People are the Key to Success

40

Highly skilled and motivated people make a difference for achieving success.

Good people can overcome many challenges (poor tools, process problems, and other surprises).

END

A SAMPLING OF ISSUES FROM 2018

42

▪ PSA Airlines, a wholly owned subsidiary of American Airlines, experienced a problem with its crew scheduling and tracking system that led to nearly 3,000 flights being cancelled over seven days in June, and cost the airline an estimated US $35 million.

▪ An air traffic control computer failure at the Eurocontrol centre in Brussels delayed an estimated 14,000 European flights in April, while over New Year’s —and for a second year in a row—the U.S. Customs and Border Protectioncomputer systems experienced an outage, leaving thousands of international passengers across the country in long queues waiting to clear customs.

▪ A coding error with the spot-welding robots at Subaru’s Indiana Automotive plant in Lafayette, Ind., meant 293 of its new Subaru Ascents had to be sent to the car crusher.

▪ Australia’s telco Telstra suffered a software problem in May that took down its 3G and 4G services nationwide for millions of its customers.

▪ Telco supplier Ericsson’s expired software certificate took down networks in 11 countries in December for nearly a day, causing major headaches for 30 million Softbank mobile customers in Japan and 25 million U.K. O2 mobile customers.

Taken from https://spectrum.ieee.org/riskfactor/computing/it/it-failures-2018-all-the-old-familiar-faces

EXAMPLES CONTINUED

▪ Google decided in October to close its Google Plus social network because of a security flaw.

▪ Facebook acknowledged in April that 87 million users had their personal data improperly accessed by now-defunct data mining company Cambridge Analytica, and later admitted in September to a “security issue” that exposed 50 million accounts to cybercriminals. In December, Facebook announced that a “bug” allowed possible unauthorized access to 6.8 million users’ photos.

▪ A December report [PDF] released by the U.S. House Oversight and Government Reform Committee into Equifax’s massive 2017 data breach stated that the breach was “entirely preventable” if the company had followed basic IT security practices. However, the report said, Equifax didn’t follow those procedures because the company had little understanding of its own IT systems or the security threat to them.

Taken from https://spectrum.ieee.org/riskfactor/computing/it/it-failures-2018-all-the-old-familiar-faces

EXAMPLES CONTINUED

▪ Canada’s federal Phoenix payroll system that was disastrously introduced in February 2016 continues to be an “incomprehensible failure” with no replacement in sight.

▪ Minnesota’s multi-year effort to deliver a fully functional vehicle and driver services system continues to have annoying problems

▪ Florida’s effort to upgrade a major roadway vehicle tolling system in June is still trying to correct numerous billing problems six months later.

▪ The U.S. Department of Veterans Affairs so thoroughly mangled the planned year-long computer implementation of the new 2017 veteran education benefits law that the VA admits it will have to start over and take yet another year to implement the law.

▪ The U.S. Internal Revenue Service suffered a major April tax day outage

▪ … Australia’s taxation commissioner told businesses to quit whining about the agency’s multiple tax system outages since they were “a fact of modern life.”

The commissioner also told businesses to expect more outages in the future.

Taken from https://spectrum.ieee.org/riskfactor/computing/it/it-failures-2018-all-the-old-familiar-faces